Multi-Campus ICT Equipment Virtualization Architecture
Multi-campus ICT equipment virtualization architecture for cloud and NFV integrated service
Abstract- We propose a virtualization architecture for multicampus
information and communication technology (ICT)
equipment with integrated cloud and NFV capabilities. The
aim of this proposal is to migrate most of ICT equipment on
campus premises into cloud and NFV platforms. Adopting this
architecture would make most of ICT services secure and
reliable and their disaster recovery (DR) economically
manageable.
We also analyze a cost function and show cost advantages of
this proposed architecture, describe implementation design
issues, and report a preliminary experimentation of NFV DR
transaction. This architecture would encourage academic
institutes to migrate their own ICT systems located on their
premises into a cloud environments.
Keywords; NFV, Data Center Migration, Disaster Recovery,
Multi-campus network
I. INTRODUCTION
There are many academic institutions that have multiple
campuses located in different cities. These institutions need
to provide information and communication technology (ICT)
services, such as E-learning services, equally for all students
on each campus. Usually, information technology (IT)
infrastructures, such as application servers, are deployed at a
main campus, and these servers are accessed by students on
each campus. For this purpose, each local area network
(LAN) on each campus is connected to a main campus LAN
via a virtual private network (VPN) over a wide area
network (WAN). In addition, Internet access service is
provided to all students on the multi-campus environment.
To access the Internet, security devices, such as firewalls and
intrusion detection systems (IDSs), are indispensable as they
protect computing resources from malicious cyber activities.
With the emergence of virtualization technologies such
as the cloud computing[1] and network functions
virtualization (NFV)[2], [3], we expected that ICT
infrastructures such as compute servers, storage devices, and
network equipment can be moved from campuses to
datacenters (DCs) economically. Some organizations have
begun to move their ICT infrastructures from their own
premises to outside DCs in order to improve security,
stability, and reliability. Also, there are a lot of contributions
to archiving DR capabilities with cloud technologies [4], [5],
[6]. Active-passive replication or active-active replication are
expected techniques that archive DR capabilities. In these
replications, a redundant backup system is required
dedicatedly at a secondary site. With migration recovery [4],
these backup resources can be shared among many users.
These studies mainly focus on the application servers. While,
integrated DR capability for ICT infrastructures, both
application and network infrastructures, are still immature.
We propose a multi-campus ICT equipment virtualization
architecture for integrated cloud and NFV capabilities. The
aim of this proposal is to migrate entire ICT infrastructures
on campus premises into cloud and NFV platforms.
Adopting this architecture for multi-campus networks would
improve access link utilization, security device utilization,
network transmission delay, disaster tolerance, and
manageability at the same time.
We also analyze the cost function and show cost
advantages of this proposed architecture.
To evaluate the feasibility of our proposed architecture,
we built a test bed on SINET5 (Science Information
NETwork 5) [7], [8], [9]. We describe the test-bed design,
and preliminary experimentation on reducing the recovery
time of VNF is reported.
The rest of this paper is organized as follows. Section II
shows background of this work. Section III shows proposed
multi-campus network virtualization architecture. Section IV
shows an evaluation of the proposed architecture in terms of
cost advantages and implementation results. Section V
concludes the paper, and future work is discussed
II. BACKGROUND OF THIS WORK
SINET5 is a Japanese academic backbone network for
about 850 research institutes and universities and provide
network services to about 30 million academic users.
SINET5 was wholly constructed and put into operation in
April 2016. SINET5 plays an important role in supporting a
wide range of research fields that need high-performance
connectivity, such as high-energy physics, nuclear fusion
science, astronomy, geodesy, seismology, and computer
science. Figure 1 shows the SINET5 architecture. It provides
points of presence, called “SINET-data centers” (DCs), and
SINET DCs are deployed in each prefecture in Japan. On
each SINET DC, an internet protocol (IP) router, MPLS-TP
system, and ROADM are deployed. The IP router
accommodates access lines from research institutes and
universities. All Every pairs of internet protocol (IP) routers
are connected by a paier of MPLS-TP paths. These paths
achieves low latency and high reliability. The IP routers and
MPLS-TP systems are connected by a 100-Gbps-based
optical path. Therefore, data can be transmitted from a
SINET DC to another SINET DC in up to 100 Gbps
throughput. In addition, users, who have 100 Gpbs access
lines, can transmit data to other users in up to 100 Gbps
throughput.
Currently, SINET5 provides a direct cloud connection
service. In this service, commercial cloud providers connect
their data centers to the SINET5 with high-speed link such as
10 Gbps link directly. Therefore, academic users can access
cloud computing resources with very low latency and high
bandwidth via SINET5. Thus, academic users can receive
high-performance computer communication between
campuses and cloud computing resources. Today, 17 cloud
service providers are directly connected to SINET5 and more
than 70 universities have been using cloud resources directly
via SINET5.
To evaluate virtual technologies such as cloud computing
and NFV technologies, we constructed at test-bed platform
(shown as “NFV platform” in fig. 1) and will evaluate the
network delay effect for ICT service with this test bed. NFV
platform are constructed at four SINET-DCs on major cities
in Japan: Sapporo, Tokyo, Osaka, and Fukuoka. At each site,
the facilities are composed of computing resources, such as
servers and storages, network resources, such as layer-2
switches, and controllers, such as NFV orchestrator, and
cloud controller. The layer-2 switch is connected to a
SINET5 router at the same site with high speed link,
100Gbps. The cloud controller configures servers and
storages and NFV orchestrator configures the VNFs on NFV
platform.
And user can setup and release VPNs between
universities, commercial clouds and NFV platforms
dynamically over SINET with on-demand controller. This
on-demand controller setup the router with NETCONF
interface. Also, this on-demand controller setup the VPN corelated
with NFV platform with REST interface.
Today there are many universities which has multiple
campus deployed over wide area. In this multi-campus
university, many VPNs (VLANs), ex hundreds of VPNs, are
desired to be configured over SINET to extend inter-campus
LAN. In order to satisfy this demand, SINET starts new
VPN services, called virtual campus LAN service. With this
service, layer 2 domains of multi-campus can be connected
as like as layer 2 switch using preconfigured VLAN rages
(ex. 1000-2000).
III. PROPOSED MULTI-CAMPUS ICT EQUIPMENT
VIRTUALIZATION ARCHITECTURE
In this section, the proposed architecture is described.
The architecture consists of two parts. First, we describe the
network architecture and clarify the issues with it. Next, a
NFV/cloud control architecture is described.
A. Proposed multi-campus network architecture
Multi-campus network architecture is shown in Figure 2.
There are two legacy network architectures and a proposed
network architecture. In legacy network architecture 1 (LA1),
Internet traffic for multiple campuses is delivered to a main
campus (shown as a green line) and checked by security
devices. After that, the internet traffic is distributed to each
campus (shown as a blue line). ICT Applications, such as Elearning
services, are deployed in a main campus and access
traffic to ICT application is carried by VPN over SINET
(shown as a blue line). In legacy network architecture 2
(LA2), the Internet access is different from LA1. The
Internet access is directly delivered to each campus and
checked by security devices deployed at each campus. In the
proposed architecture (PA), the main ICT application is
moved from a main campus to an external NFV/cloud DC.
Thus, students on both main and sub-campuses can access
ICT applications via VPN over SINET. Also, internet traffic
traverses via virtual network functions (VNFs), such as
virtual routers and virtual security devices, located at
NFV/cloud DCs. Internet traffic is checked in virtual security
devices and delivered to each main/sub-campus via VPN
over SINET.
There are pros and cons between these architectures.
Here, they are compared across five points: access link
utilization, security device utilization, network transmission
delay, disaster tolerance, and manageability.
(1) Access link utilization
The cost of an access link from sub-campus to WAN is
same in LA1, LA2 and PA. While, the cost of an access link
from a main campus to WAN of LA1 is larger than LA2 and
PA because redundant traffic traverses through the link.
While, in PA, an additional access link from a NFV/cloud
DC to WAN is required. Thus, evaluating the total accesslink
cost is important. In this evaluation, it is assumed that
additional access links from NFV/cloud DCs to WAN are
shared among multiple academic institutions who use the
NFV/cloud platform and that the cost will be evaluated
taking this sharing into account.
(2) Security device utilization
LA1 and PA is more efficient than LA2 because Internet
traffic is concentrated in LA1 and PA and a statistically
multiplexed traffic effect is expected.
In addition to it, in PA, the amount of physical
computing resources can be suppressed because virtual
security devices share physical computing resources among
multiple users. Therefore, the cost of virtual security devices
for each user will be reduced.
(3) Network transmission delay
Network delay due to Internet traffic with LA1 is longer
than that with LA2 and PA because Internet traffic to subcampuses
is detoured and transits at the main campus in LA1,
however, in LA2, network delay of Internet to sub-campuses
is directly delivered from an Internet exchange point on a
WAN to the sub-campus, so delay is suppressed. In PA,
network delay can be suppressed because the NFV and cloud
data center can be selected and located near an Internet
access gateway on WAN.
While, the network delay for ICT application services
will be longer in PA than it in LA1 and LA2. Therefore, the
effect of a longer network delay on the quality of IT
application services has to be evaluated.
(4) Disaster tolerance
Regarding Internet service, LA1 is less disaster tolerant
than LA2. In LA1, when a disaster occurs around the main
campus and the network functions of the campus go down,
students on the other sub-campuses cannot access the
internet at this time.
Regarding IT application service, IT services cannot be
accessed by students when a disaster occurs around the main
campus or data center. While, in PA, NFV/cloud DC is
located in an environment robust against earthquakes and
flooding. Thus, robustness is improved compared with LA1
and LA2.
Today, systems capable of disaster recovery (DR) are
mandatory for academic institutions. Therefore, service
disaster recovery functionality is required. In PA, back up
ICT infrastructures located at a secondary data center can be
shared with another user. Thus, no dedicated redundant
resources are required in steady state operation, so the
resource cost can be reduced. However, if VM migration
cannot be fast enough to continue services, active-passive or
active-passive replication have to be adopted. Therefore,
reducing recovery time is required to adapt migration
recovery to archive DR manageability more economically
(5) Manageability
LA1 and PA is easier to manage than LA2. Because
security devices are concentrated at a site (a main campus or
NFV/cloud data center), the number of devices can be
reduced and improving manageability.
There are three issues to consider when adopting the PA.
ï‚ž Evaluating the access link cost of an NFV/cloud
data center.
ï‚ž Evaluating the network delay effect for ICT services.
ï‚ž Evaluating the migration period for migration
recovery replication.
B. NFV and cloud control architecture
For the following two reasons, there is strong demand to
use legacy ICT systems continuously. Thus, legacy ICT
systems have to be moved to NFV/cloud DCs as virtual
application servers and virtual network functions. One reason
is that institutions have developed their own legacy ICT
systems on their own premises with vender specific features.
The second reason is that an institution’s work flows are not
easily changed, and the same usability for end users is
required. Therefore, their legacy ICT infrastructures
deployed on a campus premises should be continuously used
in the NFV/cloud environment. In the proposed multicampus
architecture, these application servers and network
functions are controlled by using per-user orchestrators.
Figure 3 shows the proposed control architecture. Each
institution deploys their ICT system on IaaS services. VMs
are created and deleted through the application interface
(API), which is provided by IaaS providers. Each institution
sets up an NFV orchestrator, application orchestrator, and
management orchestrator on VMs. Both active and standby
orchestrators are run in primary and secondary data centers,
respectively, and both active and standby orchestrators check
the aliveness of each other. The NFV orchestrator creates the
VMs and installs the virtual network functions, such as
routers and virtual firewalls, and configures them. The
application orchestrator installs the applications on VMs and
sets them up. The management orchestrator registers these
applications and virtual network functions to monitoring
tools and saves the logs outputted from the IT service
applications and network functions.
When an active data center suffers from disaster and the
active orchestrators go down, the standby orchestrators
detect that the active orchestrators are down. They start
establishing the virtual network functions and application
and management functions. After that, the VPN is connected
to the secondary data center being co-operated with the VPN
controller of WAN.
In this architecture, each institution can select NFV
orchestrators that support a user’s legacy systems.
IV. EVALUATION OF PROPOSED NETWORK ARCHITECTURE
This section details an evaluation of the access link cost
of proposed network architecture. Also, the test-bed
configuration is introduced, and an evaluation of the
migration period for migration recovery is shown.
A. Access link cost of NFV/cloud data center
In this sub-section, an evaluation of the access link cost
of PA compared with LA1 is described.
First, the network cost is defined as follows.
There is an institution, u, that has a main campus and nu
sub-campuses.
The traffic amount of institution u is defined as follows
different sites can be connected between a user site and cloud
sites by a SINET VPLS (Fig. 7). This VPLS can be
dynamically established by a portal that uses the REST
interface for the on-demand controller. For upper-layer
services such as Web-based services, virtual network
appliances, such as virtual routers, virtual firewalls, and
virtual load balancers, are created in servers through the
NFV orchestrater. DR capabilities for NFV orchestrator is
under deployment.
C. Migiration period for disaster recovery
We evaluated the VNF recovering process for disaster
recovery. In this process, there are four steps.
ï‚· Step 1: Host OS installation
ï‚· Step 2: VNF image copy
ï‚· Step 3: VNF configuration copy
ï‚· Step 4: VNF process activation
This process is started from the host OS installation because
there are VNFs that are tightly coupled with the host OS and
hypervisor. There are several kinds and versions of host OS,
so the host OS can be changed to suite to the VNF. After
host OS installation, VNF images are copied into the created
VMs. Then, the VNF configuration parameters are adjusted
to the attributions of the secondary data center environment
(for example, VLAN-ID and IP address), and the
configuration parameters are installed into VNF. After that,
VNF is activated.
In our test environment, a virtual router can be recovered
from the primary data center to the secondary data center,
and the total duration of recovery is about 6 min. Each
duration of Steps 1-4 is 3 min 13 sec, 3 min 19 sec, 11 sec,
and 17 sec, respectively.
To shorten the recovery time, currently, the standby VNF
is able to be pre-setup and activated. If the same
configuration can be applied in the secondary data center
network environment, snapshot recovering is also available.
In this case, Step 1 is eliminated, and Steps 2 and 3 are
replaced by copying a snap shot of an active VNF image,
which takes about 30 sec. In this case, the recovering time is
about 30 sec.
V. CONCLUSION
Our method using cloud and NFV functions can achieve
DR with less cost. We proposed a multi-campus equipment
virtualization architecture for cloud and NFV integrated
service. The aim of this proposal is to migrate entire ICT
infrastructures on campus premises into cloud and NFV
platforms. This architecture would encourage academic
institutions to migrate their own developed ICT systems
located on their premises into a cloud environment. Adopting
this architecture would make entire ICT systems secure and
reliable, and the DR of ICT services could be economically
manageable.
In addition, we also analyzed the cost function, and
showed a cost advantages of this proposed architecture
described implementation design issues, and reported a
preliminary experimentation of the NFV DR transaction.
REFERENCES