=======================
Working with NetDevices
=======================
`~trigger.netdevices.NetDevices` is the core of Trigger's device interaction. Anything that
communicates with devices relies on the metadata stored within `~trigger.netdevices.NetDevice`
objects.
.. contents::
:local:
:depth: 2
Your Source Data
================
Before you can work with device metadata, you must tell Trigger how and from
where to read it. You may either modify the values for these options within
``settings.py`` or you may specify the values as environment variables of the
same name as the configuration options.
Please see :doc:`configuration` for more information on how to do this. There
are two configuration options that facilitate this:
::setting:`NETDEVICES_FILE`:
The location of the file containing the metadata. Default:
`/etc/trigger/netdevices.xml`
::setting:`NETDEVICES_FORMAT`:
The format of the metadata file. Default: `xml`
When you instantiate `~trigger.netdevices.NetDevices` the specified file is
read and parsed using the specified format. The currently accepted formats are:
+ JSON
+ RANCID
+ Sqlite
+ XML
Except when using RANCID as a data source, the contents of your source data
should be a dump of relevant metadata fields from your CMDB.
If you don't have a CMDB, then you're going to have to populate this file
manually. But you're a Python programmer, right? So you can come up with
something spiffy!
Importing from RANCID
---------------------
.. versionadded:: 1.2
Experimental support for using a `RANCID `_
repository to populate your metadata is now working. We say it's experimental
because it is not yet complete. Currently all it does for you is populates the
bare minimum set of fields required for basic functionality.
To learn more please visit the section on working with the :ref:`RANCID format
`.
Supported Formats
=================
.. _xml-format:
XML
---
XML is the slowest method supported by Trigger, but it is currently the default
for legacy reasons. At some point we will switch JSON to the default.
Here is a sample what the ``netdevices.xml`` file bundled with the Trigger
source code looks like:
.. code-block:: xml
PRODUCTION
0000012345
tacacs
0101010101
1234578
Data Center
16ZZ
ROUTER
boguspassword
2010-07-19 19:56:32.0
1
1
1
INSTALLED
M40 INTERNET BACKBONE ROUTER
JUNIPER
M40-B-AC
test1-abc.net.aol.com
nobody@aol.net
17
Data Center
Data Center
C
ts1.oob.aol.com
ts1
5
5005
MONITORED
12345678 - Network Engineering
Test Lab
CR10
987654321
LAB
...
Please see ``conf/netdevices.xml`` within the Trigger source distribution for a full example.
.. _json-format:
JSON
----
JSON is the fastest method supported by Trigger. This is especially the case if
you utilize the optional C extension of `simplejson
`. The file can be minified and does
not need to be indented.
Here is a sample of what the ``netdevices.json`` file bundled with the Trigger
source code looks like (pretty-printed for readabilty):
.. code-block:: javascript
[
{
"adminStatus": "PRODUCTION",
"enablePW": "boguspassword",
"OOBTerminalServerTCPPort": "5005",
"assetID": "0000012345",
"OOBTerminalServerNodeName": "ts1",
"onCallEmail": "nobody@aol.net",
"onCallID": "17",
"OOBTerminalServerFQDN": "ts1.oob.aol.com",
"owner": "12345678 - Network Engineering",
"OOBTerminalServerPort": "5",
"onCallName": "Data Center",
"nodeName": "test1-abc.net.aol.com",
"make": "M40 INTERNET BACKBONE ROUTER",
"budgetCode": "1234578",
"budgetName": "Data Center",
"operationStatus": "MONITORED",
"deviceType": "ROUTER",
"lastUpdate": "2010-07-19 19:56:32.0",
"authMethod": "tacacs",
"projectName": "Test Lab",
"barcode": "0101010101",
"site": "LAB",
"loginPW": null,
"lifecycleStatus": "INSTALLED",
"manufacturer": "JUNIPER",
"layer3": "1",
"layer2": "1",
"room": "CR10",
"layer4": "1",
"serialNumber": "987654321",
"owningTeam": "Data Center",
"coordinate": "16ZZ",
"model": "M40-B-AC",
"OOBTerminalServerConnector": "C"
},
...
]
To use JSON, create your :setting:`NETDEVICES_FILE` full of objects that look like the
one above and set :setting:`NETDEVICES_FORMAT` to ``'json'``.
Please see ``conf/netdevices.json`` within the Trigger source distribution for
a full example.
.. _rancid-format:
RANCID
------
This is the easiest method to get running assuming you've already got a RANCID
instance to leverage. At this time, however, the metadata available from RANCID
is very limited and populates only the following fields for each
`~trigger.netdevices.Netdevice` object:
:nodeName:
The device hostname.
:manufacturer:
The representative name of the hardware manufacturer. This is also used to
dynamically populate the ``vendor`` attribute on the device object
:vendor:
The canonical vendor name used internally by Trigger. This will always be a
single, lowercased word, and is automatically set when a device object is
created.
:deviceType:
One of ('SWITCH', 'ROUTER', 'FIREWALL'). This is currently a hard-coded
value for each manufacturer.
:adminStatus:
If RANCID says the device is ``'up'``, then this is set to
``'PRODUCTION'``; otherwise it's set to ``'NON-PRODUCTION'``.
The support for RANCID comes in two forms: single or multiple instance.
Single instance is the default and expects to find the ``router.db`` file and
the ``configs`` directory in the root directory you specify.
Multiple instance will instead walk the root directory and expect to find
``router.db`` and ``configs`` in each subdirectory it finds. Multiple instance
can be toggled by seting the value of :setting:`RANCID_RECURSE_SUBDIRS` to
``True`` to your ``settings.py``.
To use RANCID as a data source, set the value of :setting:`NETDEVICES_FILE` in
``settings.py`` to the absolute path of location of of the root directory where
your RANCID data is stored and set the value :setting:`NETDEVICES_FORMAT` to
``'rancid'``.
.. note::
Make sure that the value of :setting:`RANCID_RECURSE_SUBDIRS` matches the RANCID
method you are using. This setting defaults to ``False``, so if you only
have a single RANCID instance, there is no need to add it to your
``settings.py``.
Lastly, to illustrate what a `~trigger.netdevices.NetDevice` object that has
been populated by RANCID looks like, here is the output of ``.dump()``::
Hostname: test1-abc.net.aol.com
Owning Org.: None
Owning Team: None
OnCall Team: None
Vendor: Juniper (juniper)
Make: None
Model: None
Type: ROUTER
Location: None None None
Project: None
Serial: None
Asset Tag: None
Budget Code: None (None)
Admin Status: PRODUCTION
Lifecycle Status: None
Operation Status: None
Last Updated: None
Compare that to what a device dump looks like when fully populated from CMDB
metadata in :ref:`netdevice-info`. It's important to keep this in mind, because
if you want to do device associations using any of the unpopulated fields,
you're gonna have a bad time. This is subject to change as RANCID support
evolves, but this is the way it is for now.
.. _sqlite-format:
SQLite
------
SQLite is somewhere between JSON and XML as far as performance, but also comes
with the added benefit that support is built into Python, and you get a real
database file you can leverage in other ways outside of Trigger.
.. literalinclude:: ../../conf/netdevices.sql
:language: sql
To use SQLite, create a database using the schema provided within Trigger source
distribution at ``conf/netdevices.sql``. You will need to populate the
database full of rows with the columns above and set :setting:`NETDEVICES_FORMAT` to
``'sqlite'``.
Getting Started
===============
First things first, you must instantiate `~trigger.netdevices.NetDevices`. It
has three things it requires before you can properly do this:
1. The :setting:`NETDEVICES_FILE` file must be readable and must properly
parse using the format specified by :setting:`NETDEVICES_FORMAT` (see above);
2. An instance of Redis.
3. The path to ``autoacl.py`` must be valid, and must properly parse.
How it works
------------
The `~trigger.netdevices.NetDevices` object itself is an immutable,
dictionary-like Singleton_ object. If you don't know what a Singleton is, it
means that there can only be one instance of this object in any program. The
actual instance object itself an instance of the inner
:class:`~trigger.netdevices.NetDevices._actual` class which is stored in the
module object as ``NetDevices._Singleton``. This is done as a performance boost
because many Trigger components require a NetDevices instance, and if we had to
keep creating new ones, we'd be waiting each time one had to parse
:setting:`NETDEVICES_FILE` all over again.
Upon startup, each device object/element/row found within :setting:`NETDEVICES_FILE` is
used to create a `~trigger.netdevices.NetDevice` object. This object pulls in
ACL associations from `~trigger.acl.db.AclsDB`.
.. _Singleton: http://en.wikipedia.org/wiki/Singleton_pattern
The Singleton Pattern
~~~~~~~~~~~~~~~~~~~~~
The NetDevices module object has a ``_Singleton`` attribute that defaults to ``None``.
Upon creating an instance, this is populated with the
`~trigger.netdevices.NetDevices._actual` instance::
>>> nd = NetDevices()
>>> nd._Singleton
>>> NetDevices._Singleton
This is how new instances are prevented. Whenever you create a new reference by
instantiating NetDevices again, what you are really doing is creating a reference
to ``NetDevices._Singleton``::
>>> other_nd = NetDevices()
>>> other_nd._Singleton
>>> nd._Singleton is other_nd._Singleton
True
The only time this would be an issue is if you needed to change the actual contents
of your object (such as when debugging or passing ``production_only=False``).
If you need to do this, set the value to ``None``::
>>> NetDevices._Singleton = None
Then the next call to `~trigger.netdevices.NetDevices()` will start from scratch. Keep in mind
because of this pattern it is not easy to have more than one instance (there are
ways but we're not going to list them here!). All existing instances will
inherit the value of ``NetDevices._Singleton``::
>>> third_nd = NetDevices(production_only=False)
>>> third_nd._Singleton
>>> nd._Singleton
>>> third_nd._Singleton is nd._Singleton
True
Instantiating NetDevices
========================
Throughout the Trigger code, the convention when instantiating and referencing a
`~trigger.netdevices.NetDevices` instance, is to assign it to the variable
``nd``. All examples will use this, so keep that in mind::
>>> from trigger.netdevices import NetDevices
>>> nd = NetDevices()
>>> len(nd)
3
By default, this only includes any devices for which ``adminStatus`` (aka
administrative status) is ``PRODUCTION``. This means that the device is used
in your production environment. If you would like you get all devices regardless
of ``adminStatus``, you must pass ``production_only=False`` to the constructor::
>>> from trigger.netdevices import NetDevices
>>> nd = NetDevices(production_only=False)
>>> len(nd)
4
The included sample metadata files contains one device that is marked as
``NON-PRODUCTION``.
.. _netdevice-info:
What's in a NetDevice?
======================
A `~trigger.netdevices.NetDevice` object has a number of attributes you can use
creatively to correlate
or identify them::
>>> dev = nd.find('test1-abc')
>>> dev
Printing it displays the hostname::
>>> print dev
test1-abc.net.aol.com
You can dump the values::
>>> dev.dump()
Hostname: test1-abc.net.aol.com
Owning Org.: 12345678 - Network Engineering
Owning Team: Data Center
OnCall Team: Data Center
Vendor: Juniper (JUNIPER)
Make: M40 INTERNET BACKBONE ROUTER
Model: M40-B-AC
Type: ROUTER
Location: LAB CR10 16ZZ
Project: Test Lab
Serial: 987654321
Asset Tag: 0000012345
Budget Code: 1234578 (Data Center)
Admin Status: PRODUCTION
Lifecycle Status: INSTALLED
Operation Status: MONITORED
Last Updated: 2010-07-19 19:56:32.0
You can reference them as attributes::
>>> dev.nodeName, dev.vendor, dev.deviceType
('test1-abc.net.aol.com', , 'ROUTER')
There are some special methods to perform identity tests::
>>> dev.is_router(), dev.is_switch(), dev.is_firewall()
(True, False, False)
You can view the ACLs assigned to the device::
>>> dev.explicit_acls
set(['abc123'])
>>> dev.implicit_acls
set(['juniper-router.policer', 'juniper-router-protect'])
>>> dev.acls
set(['juniper-router.policer', 'juniper-router-protect', 'abc123'])
Or get the next time it's ok to make changes to this device (more on this
later)::
>>> dev.bounce.next_ok('green')
datetime.datetime(2011, 7, 13, 9, 0, tzinfo=)
>>> print dev.bounce.status()
red
Searching for devices
=====================
Like a dictionary
-----------------
Since the object is like a dictionary, you may reference devices as keys by
their hostnames::
>>> nd
{'test2-abc.net.aol.com': ,
'test1-abc.net.aol.com': ,
'lab1-switch.net.aol.com': ,
'fw1-xyz.net.aol.com': }
>>> nd['test1-abc.net.aol.com']
You may also perform any other operations to iterate devices as you would with
a dictionary (``.keys()``, ``.itervalues()``, etc.).
Special methods
---------------
There are a number of ways you can search for devices. In all cases, you are
returned a list.
The simplest usage is just to list all devices::
>>> nd.all()
[, ,
, ]
Using ``all()`` is going to be very rare, as you're more likely to work with a
subset of your
devices.
Find a device by its shortname (minus the domain)::
>>> nd.find('test1-abc')
List devices by type (switches, routers, or firewalls)::
>>> nd.list_routers()
[, ]
>>> nd.list_switches()
[]
>>> nd.list_firewalls()
[]
Perform a case-sensitive search on any field (it defaults to ``nodeName``)::
>>> nd.search('test')
[, ]
>>> nd.search('test2')
[]
>>> nd.search('NON-PRODUCTION', 'adminStatus')
[]
Or you could just roll your own list comprehension to do the same thing::
>>> [d for d in nd.all() if d.adminStatus == 'NON-PRODUCTION']
[]
Perform a case-INsenstive search on any number of fields as keyword arguments::
>>> nd.match(oncallname='data center', adminstatus='non')
[]
>>> nd.match(vendor='netscreen')
[]
Helper function
---------------
Another nifty tool within the module is `~trigger.netdevices.device_match`,
which returns a NetDevice object::
>>> from trigger.netdevices import device_match
>>> device_match('test')
2 possible matches found for 'test':
[ 1] test1-abc.net.aol.com
[ 2] test2-abc.net.aol.com
[ 0] Exit
Enter a device number: 2
If there are multiple matches, it presents a prompt and lets you choose,
otherwise it chooses for you::
>>> device_match('fw')
Matched 'fw1-xyz.net.aol.com'.