Transcription
Nautobot Solution Guide:Source of Truth for Network Automation
ContentsIntroductionState of the UnionEvolution of Network AutomationNetwork Automation JourneyAutomation’s Foundation – Source of Truth (SoT)Intent-Based NetworkingNautobot – Source of Truth for AutomationBackgroundOverviewNautobot Use CasesFlexible Source of Truth for NetworkingExtensible Data Platform for Network AutomationPlatform for Network Automation ApplicationsNautobot ArchitectureLinuxPythonDjangoRedisSQL DatabaseWSGI middlewareHTTP ServerLDAP and/or SSONetwork Automation with NautobotREST APIsGraphQLWebhooksData Source (Git) IntegrationData ValidationCustom FieldsUser-Defined RelationshipsMulti-Database SupportChange ManagementCustom LinksJobsExport TemplatesCustomizable StatusesNetDevOps EcosystemNautobot App PlatformNautobot Plugin SystemNautobot App Platform BenefitsNautobot Extensions & AppsChatOpsGolden Configuration (GC)Device OnboardingCapacity MetricsChange ManagementUpcoming 2626262727
Nautobot Solution Guide3IntroductionData networks continue to advance at a rapid pace. This evolution drives a persistent need for more capable operational supportmodels. As networks advance, so too must the way in which they are operated. The proliferation of devices, disparity of connections, and speed of change drive requirements for faster deployment, reduced risk, increased reliability, and enhanced visibility.It’s often said that the only constant is change, but in networking there are two constants: change and the need for accurate andaccessible data. Regardless of whether the network is provisioned via the Command Line Interface (CLI), machine-to-machineAPIs, or Artificial Intelligence (AI) / Machine Learning (ML) processes, there is a requirement for an ultimate source of truth, toprovide accurate and reliable data and documentation of the intended network design and state.This solution guide presents the case for Source of Truth as a foundational pillar of a proper network automation architecture. Ithighlights the strategy for a source of truth to power closed-loop network automation systems, commonly referred to as IntentBased Networking (IBN). Finally, Nautobot is introduced as an open source software platform for Source of Truth and network automation. Nautobot will play a critical role in network automation by meeting organizations where they are today, and then growingwith them as they advance in their network automation journey.State of the UnionEvolution of Network AutomationNetwork administration has come a long way since its roots in command-line configuration. That evolution has had several stopsalong the way, each with its own lessons learned. This section will explore the lessons learned along the journey from scripting tothe AI/ML future.Automation EvolutionAI / MLSpeed and ScaleIntent-Based ivenFaster work OrchestrationNetDevOpsOperator-DrivenNetwork AutomationTechnology nBusiness Focus@networktocode
Nautobot Solution Guide4The earliest force multiplier for network administrators was simple CLI scripting and configuration copying. With basic scriptingand what’s often been referred to as templating (network engineers creating notepad snippets of known-good configurations),simple repeatable process could easily be copied from device to device, as long as those devices were similar and utilized thesame CLI. For instance, configuration could be copied from one top-of-rack, or leaf switch to another.Scripting was a manual process, driven by the engineer or administrator. Additionally, scripting usually lacks the added logic forcritical decision making needed to make production level changes. This negates the ability to fluidly perform tasks such as checkfor device readiness, terminate a script if certain conditions occur, skip a task if unnecessary, or enforce specific team standards ina manner that is modular and that scales.With the limitations of scripting and the proliferation of APIs on network devices, network organizations and teams started toexplore and adopt DevOps practices by embracing Agile, increasing collaboration, and arguably most important, they started todissect manual workflows and think through how processes can be optimized and automated. This created the movement andapproach that is NetDevOps. Simply put, this is the application of DevOps principles, processes, and tools to the network. NetDevOps opens up the idea to look at network Infrastructure as Code (IaC). If done successfully, it offers the ability to apply the principles of Continuous Integration / Continuous Development (CI/CD) to device configuration and change workflows. This providesenhanced operational speed and reduces risk of the erroneous code or changes while improving overall network reliability. Themajor gap with NetDevOps is that it is operator driven, requires new DevOps skills, and is a fairly low-level and technical approachto manage the network.The next stop is Network-as-a-Service which introduces the concept of self-service to the network. This works well by wrappingNetDevOps processes, workflows, and tools into the right user interface (UI) exposing services to internal and external customers.It’s extremely common to consume network automation through self-service portals in ServiceNow as well as chat programs suchas Slack, Microsoft Teams, or Webex Teams.Network-as-a-Service brings us forward significantly, but it still leaves a desire for more. Self-service portals often lack ways to validate form data, and they lack a proper feedback loop. Automation tools push configuration changes, but those tools often do notstore the changed data and do little to validate or provide feedback on the change. In general, this type of automation only solvesone-half of the automation loop.Intent-Based Networking (IBN) seeks to close that automation loop. Using an IBN architecture, it becomes possible to define theintended state that the network should operate in and to continuously verify that the network is operating as expected. The intentis ‘pushed’ to the network where it will be executed upon, and telemetry is then pulled or streamed to return information which isused to verify the original intent. This closed loop automation provides continuous verification and enables auto-remediation.While it is still early, the industry recognizes networks will shift towards being AI/ML driven. Still in its infancy and primarily incommercial platforms, AI/ML is still only used for specific use-cases across wired and wireless networks from correlating videoand wireless locality data to forming a cohesive security view of a retail floor. There is no doubt AI/ML will play a key role in moreintelligent and closed-loop automation over the next 5-7 years.networktocode.com@networktocode
Nautobot Solution Guide5Network Automation JourneyLike anything in the world of technology, there is no one-size fits all network. Around the globe today, organizations rely on different combinations of all of the phases described and many others not yet discussed. In general, most organizations are embracing some form of network automation and orchestration that is typically aligned with NetDevOps or Network-as-a-Service. Theleading-edge rests firmly on IBN and on creating closed-loop systems.Operational SpeedResponsiveness to ed State Device Driven CLI/GUI Managed Limited Visibility Device Driven Script Provisioned Limited Visibility Operator Driven Automated Management Moderate Visibility Self-Service Orchestration Driven Moderate VisibilityIntent-BasedNetworking Event-Driven Closed-Loop Automation Enhanced VisibilituyHaving a vision that aligns to closed-loop systems, or IBN, is strong. IBN systems provide the business with responsive capabilitiesand operational speed made possible by increased reliability and security that identifies and reacts to anomalies and threats. WithIBN, the shift moves to an event-driven system. IBN reduces operational overhead, increases operational visibility, and ties networkpolicy and delivery closer together with the applications it exists to serve.Underneath each of these models: manual, scripted, NetDevOps, as-a-service, and Intent-Based Networking lies one common challenge. That challenge is data. This comes in the form of managing knowledge and data, which is often documentation. In general,network documentation is extremely lacking. When it exists, it exists across multiple sources. It is rarely up-to-date and morerarely trusted internally. This lack of a central authority for network information such as VLANs, IP Addresses, inventory, cabling,etc. leads to a plethora of issues and wasted time. What is needed is a centralized network source of truth.Automation’s Foundation – Source of Truth (SoT)A reliable Source of Truth (SoT) is critical at each step of the network automation journey. Having accurate and accessible networkdata is the foundational element to enable success whether an organization is moving from manual config to Network-as-a-Service or from scripting to NetDevOps.This makes SoT a natural starting point when embarking on a network automation journey. Deploying a centralized SoT prior to, oralongside a network automation project helps to ensure that at a minimum, data has a home. Because many network automationprojects are undertaken alongside equipment refresh, modernization, or transformation initiatives, the timing is perfect.networktocode.com@networktocode
Nautobot Solution Guide6SoT Applied to Automation JourneyOperational SpeedResponsiveness to ed State Device Driven CLI/GUI Managed Limited VisibilitySource-of-Truth Device Driven Script Provisioned Limited Visibility Operator Driven Automated Management Moderate Visibility Self-Service Orchestration Driven Moderate VisibilityIntent-BasedNetworking Event-Driven Closed-Loop Automation Enhanced VisibilituyData Requirements:InventoryManagementIP AddressManagementVLANManagement Tenants Regions Sites Devices Device Component Aggregates IRs Prefixes IP Addresses VRFs VLANs VLAN GroupsCablingManagementCircuitManagement Circuits Providers Types Circuits Providers TypesRacks ElevationDiagramsPower Feeds PanelsConfigurationData Interfaces Routing ExtensibleThe SoT provides a single source of information for network and automation data. The SoT ensures network designs, standards,and configurations are defined in a vendor-neutral way to enable data-driven network automation at scale. This data is what drivesthe network.Any data can be stored within the Source of Truth including inventory, IP addresses, VLANs, VRFs, BGP ASNs, interfaces, connections, power feeds, circuits, and much more. The source of truth should be extensible for future data requirements that go wellbeyond what may be initially required “out of the box.”The further along the automation journey one goes, the more critical the SoT becomes. Although dangerous, manual configoperations or transactional-based automation could theoretically skip a SoT altogether. At the other extreme, closed-loop systemsrequire a source of truth as a central repository for defining the network intent.Intent-Based NetworkingTo understand why IBN requires a SoT, one must first understand the concepts of intent based networks. The concept of ‘Intent’has roots in Promise Theory, where intent-based systems push an ‘intent’ and assume the intent is carried out. This ‘push’ isknown as ‘intended state’ (often in the form of configuration). Devices will then report back ‘actual state’ (often in the form of operational state) which can be verified against the intent.networktocode.com@networktocode
Nautobot Solution Guide7One example of intended state would be configuration for a single switch. The configuration could be a small snippet or a fullconfiguration. That intent would be pushed down and assumed to be carried out. This means the system providing intent doesnot need to do any verification prior to sending intent, nor is it required to wait and verify success. The device that receives intentis responsible for attempting to implement the intent and reporting back actual state. If the state can’t be applied, then the actualstate will fail verification at the point of origin. This creates a closed loop automation system known as an intent loop.IntentSystemActual StateIntended StateNetworkDeviceThe intent system requires a SoT to define intent or desired state. As actual state is reported back, it is compared against theintended state within the SoT. This comparison provides the detail required to know if the operating or actual state is implementedas documented and designed. The SoT is central to the intent loop.At a 50,000-foot view, the SoT becomes the fulcrum on which a network automation architecture pivots. For example, the Networkto Code Reference Network Automation Architecture is shown gDocumentationSource of TruthDeviceInventoryDCIMNetwork DevicePropertiesIPAMConfigurationTemplatesSource of Truth AggregationAutomation EngineConfigurationRenderingTelemetry and AnalyticsZTPOrchestration sioningCollectorContinuous Integration/Continuous Deployment (CI/CD)Continuous Integration/Continuous Deployment (CI/CD)User InteractionsChangeManagementNetworkThe architecture above provides a holistic, closed-loop automation system with modularity and tool choice in mind along withintegrating into DevOps-powered toolchains. Most notable within the architecture is the central role of the Source of Truth. It isthe heart of the Network to Code Reference Network Automation Architecture. It becomes the authoritative source for intent. It isthe place to define network design and configuration, the source for enabling compliance of network state, and a source of dataenrichment for telemetry systems.networktocode.com@networktocode
Nautobot Solution Guide8Nautobot – Source of Truth for AutomationBackgroundIn recent years, NetBox, an open source project, gained adoption as an IP Address Management (IPAM) and Data Center Infrastructure Management (DCIM) platform while also serving as a Source of Truth for documenting networks and network automation solutions. Network to Code has embraced the popular open source project, positioning and implementing it as the Source ofTruth within many enterprise network automation systems. Over time, after amassing extensive experience with Source of Truthplatforms, such as NetBox, coupled with frequent collaboration with user communities and many enterprise customers on theirevolving requirements, Network to Code concluded that the Source of Truth should evolve as well.The next generation of the SoT is one that is feature rich, flexible, extensible, and that serves as a platform for easily enablingcustom automation applications to meet the needs of enterprise network operators. It is with great pleasure that we announceNautobot, our next-generation Network Source of Truth and Network Automation Platform.OverviewNautobot is an open source software (OSS) Source of Truth and Network Automation Platform. Built on top of a fork of NetBoxv2.10.4, Nautobot allows network organizations to streamline the way data is managed and integrated while serving as the foundation to enable network automation through its flexibility, extensibility, and robust set of APIs. It is a vendor-agnostic, networkcentric Configuration Management Data Base (CMDB) that allows users to store common data constructs and models such asIP Addresses and VLANs. It also allows for the customization of data models, creation of new data models, and serves as an appplatform for network automation applications that leverage the rich data stored within the source of truth database.Nautobot is the cornerstone of anynetwork automation strategy.While Nautobot can be used to kickstart an automation project’s migration from countless and distributed spreadsheets to a centralized Source of Truth, it also offers great flexibility and extensibility that can be leveraged to power network automation solutions.Nautobot Extensible Data PlatformREST ExtensibleCustomerExtensionCustomerExtensionGolden ConfigServiceNow SyncChatOpsCapacity MetricsActual StateClosed-LoopAutomatedGraphQL APIIntegratedOrchestratorIntended StateChange MgmtOnboardingFuture ExtensionCustomer ExecutorContinuouslyVerifiedOpen-SourceIntegrated
Nautobot Solution Guide9One example of that extensibility is Nautobot’s native integration with Git. This integration seamlessly aggregates structured datafiles with the native data already supported by Nautobot. The following graphic represents a short list of some of those native datamodels supported by Nautobot:Inventory Management Tenants Regions Sites Devices Device ComponentsCablingvCabling Connections Tracing PathsCircuit ManagementIP Address Management Circuits Providers Types Aggregates Prefixes IP Addresses VRFs Route TargetsPower Feeds PanelsVLAN Management VLANs VLAN GroupsNautobot Use CasesWhile a wide array of use cases exists for Nautobot, the following three core use cases drive product development: Flexible Source of Truth for Networking – is the core data storage feature set, but with maximum flexibility that allows usersto augment existing data models, define relationships between models, and define custom business rules so only good data isever added to Nautobot. Extensible Data Platform for Network Automation – is how Nautobot powers network automation architectures by way of itsRESTful and GraphQL APIs while also enabling the aggregation of disparate data sources and creating a unified view and APIinto that data. Platform for Network Automation Applications – re-defines how network automation applications or “apps” are built anddeployed. Reducing development time by nearly 70%, users can create their own extensions and apps that cater to theirunique requirements.These three core use cases are further described below.Flexible Source of Truth for NetworkingNautobot is an extremely flexible and extensible SoT for networking organizations and teams. Think of the core functionality provided by Nautobot as a network-centric CMDB that is purpose-built for network teams to manage on a day-to-day basis.Initial deployments of Nautobot usually include the migration of data sets that exist in a number of distributed spreadsheets andare rarely trusted within an organization. Nautobot becomes a centralized and trusted Source of Truth. It enables proper datahygiene and empowers users to define business rules regarding the types of data that are allowed to be stored within Nautobot.networktocode.com@networktocode
Nautobot Solution Guide10Some organizations have already deployed trusted sources of data, referred to as a System of Record (SoR), that serve as anauthoritative source for specific data types. When these trusted SoRs already exist, it is possible to integrate them with Nautobot’sextensibility system. This design allows Nautobot to provide a consolidated front-end interface and to serve as a Single Source ofTruth (SSoT) by creating a source of truth aggregation layer for all data sources that exist in a given environment.For example, it is not uncommon to have an existing trusted System of Record for circuits and circuit contracts. If that’s the case,then that data can be synchronized with Nautobot to provide a holistic view of all network data in a single view.Within the core of the platform, it is quite common to start populating device inventory, rack elevations, cabling, and IP addressingto get started. This provides solid network documentation for the network team. For example, rack elevations and cabling becomecritical when planning refresh and greenfield deployments. Nautobot offers a structured way to document hardware for networkand operations teams while being the foundation to eventually drive network automation.networktocode.com@networktocode
Nautobot Solution Guide11Nautobot provides change and workflow management native in the platform. This allows users to stage and review changes before committing them. These capabilities are detailed below in the Nautobot App Platform section.Nautobot utilizes a robust set of view controls based on permissions. Administrators have options to remove and hide what userssee based on what information they are authorized to view.While simply storing the data and accessing it via the native UI is the first step, simplifying access to the data further enablesconsumption and adoption of Nautobot within an organization. For example, consider deploying Nautobot’s Chatbot applicationto access data. Chatbot offers several advantages to drive collaboration and more efficient troubleshooting. See the NautobotApplications section for more details.networktocode.com@networktocode
Nautobot Solution Guide12ChatOpsAccess network data andperform common tasks from chatMulti-platform ChatbotAutomate network systems and devicesAdd your own chat commandsExtensible Data Platform for Network AutomationThe lack of availability of network data is a critical challenge in network automation. The primary reason most organizations findnetwork automation to be challenging is due to a lack of data management and a failure to embrace a Source of Truth-first strategy.Availability of network data is critical to drive network policy and configuration. One tell-tale sign that availability is missing is whenengineers have to login to the actual network device to retrieve its configuration rather than trusting existing data sources anddocumentation. This scenario is the opposite of intent-based networks because only the actual configured state is retrieved – butwhere was the intended state retrieved for comparison? A far better solution is to hold a trusted intended state within Nautobotand then compare its data with the actual state as retrieved from the device.DocumentationAny ToolSpreadsheets andDisparate ToolsIntended StateDataNetwork Automation Journeynetworktocode.com@networktocodeData-Driven NetworkAutomation
Nautobot Solution Guide13As an extensible data platform, Nautobot is much more than visual documentation. Nautobot has integrations and extensibilityto aggregate disparate data sources and streamline the consumption of data across them. Nautobot offers robust platform APIsincluding a traditional RESTful (HTTP) API, webhooks that may be triggered when any object changes within the system, andGraphQL—a modern approach to APIs that integrate traditional HTTP APIs and database queries. GraphQL allows users to selectively query exactly what is needed from the system within a single API call.Nautobot ships with standard data models to get new users started quickly. When additional data types are needed, it is possibleto build custom native models or to use Git-stored YAML files which are dynamically rendered in Nautobot as structured data. Withthis integration, it is possible to query Nautobot’s REST and GraphQL APIs that aggregate native data and YAML data for a givenregion, site, or device. This works well for static data such as NTP, SNMP, AAA, and other global services across the network.Beyond what the platform offers natively, there are additional NetDevOps integrations such as an Ansible collection of modules toperform CRUD operations and dynamic inventory and a separate dynamic inventory that is purpose-built for use with Nornir.Platform for Network Automation ApplicationsThe third and final core Nautobot use case is to provide a platform for network automation applications. As a single Source ofTruth, Nautobot is strategically positioned to provide the foundation of a closed-loop network automation system. As such, it mustbe extensible and customizable for functionality unique to a given environment.OS UpgradesBackupsComplianceYour AppApp NNetwork Automation AppsUI ElementsAPIData ModelsLightweigth ExtensionsNautobot Python APIThe Nautobot App Platform provides an architecture from which network automation applications can be built. Nautobot Apps areflexible and customized additions that can be as lightweight or solution centric as needed. This enables users to easily add new UIelements or panels to customize existing web pages in Nautobot, and allows for the creation of custom APIs and data models too.Beyond basic extensions, apps can be more robust. An app could integrate with circuit databases or a ServiceNow CMDB. An appcan provide full-fledged network automation to tackle problems such as backups, OS upgrades, or firewall management. Leveraging Nautobot for creating custom apps instead of building a net new internal framework or dedicated app saves on average 6575% cost and effort, not to mention the additional resources required to maintain a pure custom application framework.networktocode.com@networktocode
Nautobot Solution Guide14Nautobot ArchitectureAt its core, Nautobot is a Source of Truth platform with an extensible plugin system that enables it to serve as a network automation platform. It’s worth understanding the key components that comprise Nautobot from a more technical and engineeringvantage point. The core architectural components of the Nautobot platform are described below.LinuxNautobot commonly runs as a Linux application, either natively or within a set of Docker containers. It could alternatively run as acloud-native application.PythonNautobot is implemented in the Python 3 programming language, which provides memory management, exception handling,object orientation, and many other built-in features. Versions 3.6 and later are supported.DjangoThe Django web framework acts as the “glue” between all of the other architectural components. It provides consistent APIs forthe HTTP server to interact with the Python application code, and for the Python code to talk to Redis and the database.RedisRedis provides features for data caching and task queueing, improving the application performance in the first case and enablingthe execution of long-running background tasks (jobs, cross-system data synchronization, etc.) in the second case.SQL DatabaseAn object-relational SQL database underlying the application provides for reliable, high-performing data storage and access. PostgreSQL is currently supported, with support for MySQL on the near-term roadmap.WSGI middlewareThis middleware layer uses uWSGI or Gunicorn between the HTTP server and the Django web framework to provide features suchas load balancing and multiprocessing.HTTP ServerThe HTTP server uses Apache or NGINX to provides an HTTP(S) socket for access to the application by users (to the Nautobotweb UI) and automated processes (to the REST and GraphQL APIs).LDAP and/or SSOAs an optional component, users can integrate with LDAP/SSO systems. By default, Nautobot uses Django’s built-in user management and authentication system. It can optionally offload user management and authentication operations to an existing remotesystem via LDAP or single sign on (SSO).networktocode.com@networktocode
Nautobot Solution Guide15Network Automation with NautobotNautobot’s critical role in network automation architectures is enabled by the first two use cases covered previously including Flexible Source of Truth for Networking and Extensible Data Platform for Network Automation.At the heart of enabling network automation, Nautobot has a robust set of platform APIs. It offers RESTful (HTTP) APIs, webhooks,a GraphQL Query engine, and native git integration to seamlessly integrate with existing YAML data files to create an aggregateview into the source of truth.This section will describe these features and more that Nautobot brings to bear for those environments focused on deliveringnetwork automation solutions.REST APIsNautobot provides a standard REST API complete with create, read, update, and delete (CRUD) operations. This is a traditional setof programmatic APIs to manage Nautobot using machine to machine APIs. The REST API can be leveraged to perform queries,but also update inventory, interfaces, cabling, and manage any other piece of data that is stored in Nautobot.This API is further extensible via the plugin system allowing users to create their own custom APIs via lightweight extensions or apps.GraphQLNautobot utilizes GraphQL to provide powerful read-only query capabilities. GraphQL is commonly thought of as a hybrid betweendatabase query language and REST API. It utilizes a defined query language along with a defined format for data requests andreceipt. This allows a client to request filtered data from a structured data source with well-defined schema. This makes SoT dataextremely accessible in simplified API calls. For example, with a single API call, it’s possible to query for all data on a given deviceor all devices, as opposed to using numerous queries using the standard REST APIs. What may take 10-12 REST API calls cannow be performed in a single GraphQL query.networktocode.com@networktocode
Nautobot Solution Guide16Nautobot’s GraphQL implementation includes a built-in, interactive, graphical GraphQL client called GraphiQL. This client allows theGraphQL API to be explored and used natively on the system.There is also a REST API endpoint that allows users to execute a GraphQL query programmatically
Finally, Nautobot is introduced as an open source software platform for Source of Truth and network au-tomation. Nautobot will play a critical role in network automation by meeting organizations where they are today, and then growing with
