Prepare release 3.0.1 (#1203)

* Bugfix/ios get interfaces counters mgmt0 (#1201) * IOS: fix get_interfaces_counters when interface = canonical Fixes #1200 mgmt0 interface is not translated to it's canonical name on the 'show interface' command. Thix fix attempt to use the expected Canonical name first and fallback to the interface name in case of a KeyError is raised * Add test * Make black happy ! * Version 3.0.1 (#1202) * Junos: close configuration in case configure_private is set to avoid configure session to stay forever. (#1199) This is done in commit_config and discard_config functions. Co-authored-by: Mircea Ulinic <mirceaulinic@users.noreply.github.com> * Bugfix/ios get optics input na (#1206) * IOS: fix get_optics when input is N/A Extend the current check to ouput value to the input value. Input value could also be N/A Fixes #1205 * Add test * Bugfix/ios get optics support vss (#1208) * IOS: support get_optics on VSS Setup Introduce a new class function: _is_vss If the current swith is a VSS setup, run the following commands an parse the output:: show interfaces transceiver switch 1 show interfaces transceiver switch 2 * Add test * Make black happy * Bump pytest from 5.4.1 to 5.4.2 (#1204) Bumps [pytest](https://github.com/pytest-dev/pytest) from 5.4.1 to 5.4.2. - [Release notes](https://github.com/pytest-dev/pytest/releases) - [Changelog](https://github.com/pytest-dev/pytest/blob/master/CHANGELOG.rst) - [Commits](pytest-dev/pytest@5.4.1...5.4.2) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> Co-authored-by: Mircea Ulinic <mirceaulinic@users.noreply.github.com> * Update README.md and setup.py so branch has some differences (#1209) * Fix cmd_verify and null character on NX-OS (#1214) * Bump ddt from 1.3.1 to 1.4.1 (#1219) Bumps [ddt](https://github.com/datadriventests/ddt) from 1.3.1 to 1.4.1. - [Release notes](https://github.com/datadriventests/ddt/releases) - [Commits](datadriventests/ddt@1.3.1...1.4.1) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> * Add eos_fn0039_config optional arg to toggle FN 0039 on config commands (#1212) Co-authored-by: Kirk Byers <ktbyers@twb-tech.com> Co-authored-by: Exane Server Team <frederic.brin@exane.com> Co-authored-by: spinoshi <simone.spinelli@geant.org> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> Co-authored-by: Kirk Byers <ktbyers@twb-tech.com>
napalm-automation · May 15, 2020 · 456f736 · 456f736
1 parent 803f764
commit 456f736
Show file tree

Hide file tree

Showing 20 changed files with 1,038 additions and 25 deletions.
diff --git a/README.md b/README.md
@@ -129,14 +129,18 @@ Podcasts
 Authors
 =======
  * David Barroso ([dbarrosop@dravetech.com](mailto:dbarrosop@dravetech.com))
+ * Mircea Ulinic ([ping@mirceaulinic.net](mailto:ping@mirceaulinic.net))
+ * Kirk Byers ([ktbyers@twb-tech.com](mailto:ktbyers@twb-tech.com))
  * Elisa Jasinska ([elisa@bigwaveit.org](mailto:elisa@bigwaveit.org))
  * Many others, check the [contributors](https://github.com/napalm-automation/napalm/graphs/contributors) page for details.
 
 
 Thanks
 ======
 
-This project was founded by David Barroso as part of [Spotify][spotify] and Elisa Jasinska as part of [BigWave IT][bigwave]. Originally it was hosted by the [Spotify][spotify] organization but due to the many contributions received by third parties we agreed creating a dedicated organization for NAPALM and give a big thanks to [Spotify][spotify] for the support.
+This project is maintained by David Barroso, Mircea Ulinic, and Kirk Byers and a set of other contributors.
+
+Originally it was hosted by the [Spotify][spotify] organization but due to the many contributions received by third parties we agreed creating a dedicated organization for NAPALM and give a big thanks to [Spotify][spotify] for the support.
 
 [spotify]: http://www.spotify.com
 [bigwave]: http://bigwaveit.org/
diff --git a/docs/support/index.rst b/docs/support/index.rst
@@ -133,6 +133,13 @@ ____________________________________
 * :code:`transport` (eos, ios, nxos) - Protocol to connect with (see `The transport argument`_ for more information).
 * :code:`use_keys` (ios, iosxr, nxos_ssh) - Paramiko argument, enable searching for discoverable private key files in ``~/.ssh/`` (default: ``False``).
 * :code:`eos_autoComplete` (eos) - Allows to set `autoComplete` when running commands. (default: ``None`` equivalent to ``False``)
+* :code:`eos_fn0039_config` (eos) - Transform old style configuration to the new 
+  style, available beginning with EOS release 4.23.0, as per FN 0039. Beware 
+  that enabling this option will change the configuration you're loading 
+  through NAPALM. Default: ``False`` (won't change your configuration 
+  commands).
+
+  .. versionadded:: 3.0.1
 
 The transport argument
 ______________________

diff --git a/napalm/eos/eos.py b/napalm/eos/eos.py
@@ -129,6 +129,7 @@ def _process_optional_args(self, optional_args):
         transport = optional_args.get(
             "transport", optional_args.get("eos_transport", "https")
         )
+        self.fn0039_config = optional_args.pop("eos_fn0039_config", False)
         try:
             self.transport_class = pyeapi.client.TRANSPORTS[transport]
         except KeyError:
@@ -285,9 +286,13 @@ def _load_config(self, filename=None, config=None, replace=True):
 
         try:
             if self.eos_autoComplete is not None:
-                self.device.run_commands(commands, autoComplete=self.eos_autoComplete)
+                self.device.run_commands(
+                    commands,
+                    autoComplete=self.eos_autoComplete,
+                    fn0039_transform=self.fn0039_config,
+                )
             else:
-                self.device.run_commands(commands)
+                self.device.run_commands(commands, fn0039_transform=self.fn0039_config)
         except pyeapi.eapilib.CommandError as e:
             self.discard_config()
             msg = str(e)

diff --git a/napalm/eos/pyeapi_syntax_wrapper.py b/napalm/eos/pyeapi_syntax_wrapper.py
@@ -32,9 +32,11 @@ def run_commands(self, commands, **kwargs):
         :param kwargs: other args
         :return: list of outputs
         """
-        if isinstance(commands, str):
-            new_commands = [cli_convert(commands, self.cli_version)]
-        else:
-            new_commands = [cli_convert(cmd, self.cli_version) for cmd in commands]
+        fn0039_transform = kwargs.pop("fn0039_transform", True)
+        if fn0039_transform:
+            if isinstance(commands, str):
+                commands = [cli_convert(commands, self.cli_version)]
+            else:
+                commands = [cli_convert(cmd, self.cli_version) for cmd in commands]
 
-        return super(Node, self).run_commands(new_commands, **kwargs)
+        return super(Node, self).run_commands(commands, **kwargs)
diff --git a/napalm/ios/ios.py b/napalm/ios/ios.py
@@ -753,21 +753,46 @@ def _send_command_postprocess(output):
         output = re.sub(r"^Time source is .*$", "", output, flags=re.M)
         return output.strip()
 
+    def _is_vss(self):
+        """
+        Returns True if a Virtual Switching System (VSS) is setup
+        """
+        vss_re = re.compile("Switch mode[ ]+: Virtual Switch", re.M)
+        command = "show switch virtual"
+        output = self._send_command(command)
+
+        return bool(vss_re.search(output))
+
     def get_optics(self):
         command = "show interfaces transceiver"
         output = self._send_command(command)
+        is_vss = False
 
         # Check if router supports the command
         if "% Invalid input" in output:
             return {}
+        elif "% Incomplete command" in output:
+            if self._is_vss():
+                is_vss = True
+                command1 = "show interfaces transceiver switch 1"
+                command2 = "show interfaces transceiver switch 2"
+                output1 = self._send_command(command1)
+                output2 = self._send_command(command2)
 
         # Formatting data into return data structure
         optics_detail = {}
 
-        try:
-            split_output = re.split(r"^---------.*$", output, flags=re.M)[1]
-        except IndexError:
-            return {}
+        if is_vss:
+            try:
+                split_output = re.split(r"^---------.*$", output1, flags=re.M)[1]
+                split_output += re.split(r"^---------.*$", output2, flags=re.M)[1]
+            except IndexError:
+                return {}
+        else:
+            try:
+                split_output = re.split(r"^---------.*$", output, flags=re.M)[1]
+            except IndexError:
+                return {}
 
         split_output = split_output.strip()
 
@@ -786,12 +811,17 @@ def get_optics(self):
 
             port_detail = {"physical_channels": {"channel": []}}
 
-            # If interface is shutdown it returns "N/A" as output power.
+            # If interface is shutdown it returns "N/A" as output power
+            # or "N/A" as input power
             # Converting that to -100.0 float
             try:
                 float(output_power)
             except ValueError:
                 output_power = -100.0
+            try:
+                float(input_power)
+            except ValueError:
+                input_power = -100.0
 
             # Defaulting avg, min, max values to -100.0 since device does not
             # return these values
@@ -2129,9 +2159,17 @@ def get_interfaces_counters(self):
                     )
                     match = re.search(regex, line)
                     if match:
-                        interface = canonical_interface_name(interface)
-                        counters[interface]["rx_discards"] = int(match.group("IQD"))
-                        counters[interface]["tx_discards"] = int(match.group("OQD"))
+                        can_interface = canonical_interface_name(interface)
+                        try:
+                            counters[can_interface]["rx_discards"] = int(
+                                match.group("IQD")
+                            )
+                            counters[can_interface]["tx_discards"] = int(
+                                match.group("OQD")
+                            )
+                        except KeyError:
+                            counters[interface]["rx_discards"] = int(match.group("IQD"))
+                            counters[interface]["tx_discards"] = int(match.group("OQD"))
 
         return counters
 

diff --git a/napalm/junos/junos.py b/napalm/junos/junos.py
@@ -283,12 +283,16 @@ def commit_config(self, message=""):
         self.device.cu.commit(ignore_warning=self.ignore_warning, **commit_args)
         if not self.lock_disable and not self.session_config_lock:
             self._unlock()
+        if self.config_private:
+            self.device.rpc.close_configuration()
 
     def discard_config(self):
         """Discard changes (rollback 0)."""
         self.device.cu.rollback(rb_id=0)
         if not self.lock_disable and not self.session_config_lock:
             self._unlock()
+        if self.config_private:
+            self.device.rpc.close_configuration()
 
     def rollback(self):
         """Rollback to previous commit."""

diff --git a/napalm/nxos_ssh/nxos_ssh.py b/napalm/nxos_ssh/nxos_ssh.py
@@ -438,13 +438,13 @@ def open(self):
     def close(self):
         self._netmiko_close()
 
-    def _send_command(self, command, raw_text=False):
+    def _send_command(self, command, raw_text=False, cmd_verify=True):
         """
         Wrapper for Netmiko's send_command method.
 
         raw_text argument is not used and is for code sharing with NX-API.
         """
-        return self.device.send_command(command)
+        return self.device.send_command(command, cmd_verify=cmd_verify)
 
     def _send_command_list(self, commands, expect_string=None):
         """Wrapper for Netmiko's send_command method (for list of commands."""
@@ -506,7 +506,7 @@ def is_alive(self):
                 return {"is_alive": False}
             else:
                 # Try sending ASCII null byte to maintain the connection alive
-                self._send_command(null)
+                self._send_command(null, cmd_verify=False)
         except (socket.error, EOFError):
             # If unable to send, we can tell for sure that the connection is unusable,
             # hence return False.

diff --git a/requirements-dev.txt b/requirements-dev.txt
@@ -1,8 +1,8 @@
 black==19.10b0
 coveralls==2.0.0
-ddt==1.3.1
+ddt==1.4.1
 flake8-import-order==0.18.1
-pytest==5.4.1
+pytest==5.4.2
 pytest-cov==2.8.1
 pytest-json==0.4.0
 pytest-pythonpath==0.7.3

diff --git a/setup.py b/setup.py
@@ -4,17 +4,22 @@
 with open("requirements.txt", "r") as fs:
     reqs = [r for r in fs.read().splitlines() if (len(r) > 0 and not r.startswith("#"))]
 
+with open("README.md", "r") as fs:
+    long_description = fs.read()
+
 
 __author__ = "David Barroso <dbarrosop@dravetech.com>"
 
 setup(
     name="napalm",
-    version="3.0.0",
+    version="3.0.1",
     packages=find_packages(exclude=("test*",)),
     test_suite="test_base",
     author="David Barroso, Kirk Byers, Mircea Ulinic",
     author_email="dbarrosop@dravetech.com, ping@mirceaulinic.net, ktbyers@twb-tech.com",
     description="Network Automation and Programmability Abstraction Layer with Multivendor support",
+    long_description=long_description,
+    long_description_content_type="text/markdown",
     classifiers=[
         "Topic :: Utilities",
         "Programming Language :: Python",

diff --git a/test/eos/test_heredoc.py b/test/eos/test_heredoc.py
@@ -57,7 +57,9 @@ def test_heredoc(self):
             "idle-timeout 15",
         ]
 
-        self.device.device.run_commands.assert_called_with(expected_result)
+        self.device.device.run_commands.assert_called_with(
+            expected_result, fn0039_transform=False
+        )
 
     def test_mode_comment(self):
         raw_config = dedent(
@@ -108,7 +110,9 @@ def test_mode_comment(self):
             "permit host 192.0.2.3",
         ]
 
-        self.device.device.run_commands.assert_called_with(expected_result)
+        self.device.device.run_commands.assert_called_with(
+            expected_result, fn0039_transform=False
+        )
 
     def test_heredoc_with_bangs(self):
 
@@ -145,4 +149,6 @@ def test_heredoc_with_bangs(self):
             "idle-timeout 15",
         ]
 
-        self.device.device.run_commands.assert_called_with(expected_result)
+        self.device.device.run_commands.assert_called_with(
+            expected_result, fn0039_transform=False
+        )
diff --git a/test/ios/mocked_data/test_get_interfaces_counters/no_canonical_for_mgmt/expected_result.json b/test/ios/mocked_data/test_get_interfaces_counters/no_canonical_for_mgmt/expected_result.json
@@ -0,0 +1,16 @@
+{
+    "mgmt0": {
+        "rx_unicast_packets": 10039584,
+        "rx_octets": 860372462,
+        "rx_broadcast_packets": 10025891,
+        "rx_multicast_packets": 6,
+        "rx_errors": 0,
+        "rx_discards": 0,
+        "tx_unicast_packets": 309048,
+        "tx_octets": 110610635,
+        "tx_broadcast_packets": -1,
+        "tx_multicast_packets": -1,
+        "tx_errors": 0,
+        "tx_discards": 0
+    }
+}
diff --git a/...mocked_data/test_get_interfaces_counters/no_canonical_for_mgmt/show_interface_summary.txt b/...mocked_data/test_get_interfaces_counters/no_canonical_for_mgmt/show_interface_summary.txt
@@ -0,0 +1,12 @@
+
+
+ *: interface is up
+ IHQ: pkts in input hold queue     IQD: pkts dropped from input queue
+ OHQ: pkts in output hold queue    OQD: pkts dropped from output queue
+ RXBS: rx rate (bits/sec)          RXPS: rx rate (pkts/sec)
+ TXBS: tx rate (bits/sec)          TXPS: tx rate (pkts/sec)
+ TRTL: throttle count
+
+  Interface                   IHQ       IQD       OHQ       OQD      RXBS      RXPS      TXBS      TXPS      TRTL
+-----------------------------------------------------------------------------------------------------------------
+* mgmt0                         0         0         0         0         0         1         0         0         0
diff --git a/test/ios/mocked_data/test_get_interfaces_counters/no_canonical_for_mgmt/show_interfaces.txt b/test/ios/mocked_data/test_get_interfaces_counters/no_canonical_for_mgmt/show_interfaces.txt
@@ -0,0 +1,31 @@
+mgmt0 is up, line protocol is up (connected)
+  Hardware is I82580 MGMT, address is 0008.e3ff.fd68 (bia 0008.e3ff.fd68)
+  Internet address is 10.211.127.1/24
+  MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
+     reliability 255/255, txload 1/255, rxload 1/255
+  Encapsulation ARPA, loopback not set
+  Keepalive not supported
+  Full-duplex, 1000Mb/s
+  Media-type configured as RJ45 connector
+  input flow-control is off, output flow-control is unsupported
+  Clock mode is auto
+  ARP type: ARPA, ARP Timeout 04:00:00
+  Last input 00:00:01, output never, output hang never
+  Last clearing of "show interface" counters 23w5d
+  Input queue: 0/1000/0/0 (size/max/drops/flushes); Total output drops: 0
+  Queueing strategy: fifo
+  Output queue: 0/40 (size/max)
+  5 minute input rate 0 bits/sec, 0 packets/sec
+  5 minute output rate 0 bits/sec, 0 packets/sec
+     10039584 packets input, 860372462 bytes, 0 no buffer
+     Received 10025891 broadcasts (416 IP multicasts)
+     0 runts, 0 giants, 0 throttles
+     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
+     0 watchdog, 0 multicast, 0 pause input
+     0 input packets with dribble condition detected
+     309048 packets output, 110610635 bytes, 0 underruns
+     0 output errors, 0 collisions, 25 interface resets
+     0 unknown protocol drops
+     0 babbles, 0 late collision, 0 deferred
+     0 lost carrier, 0 no carrier, 0 pause output
+     0 output buffer failures, 0 output buffers swapped out