A framework for IoT service experiment platforms in smart-city environments

Cities provide an excellent platform for gathering and detection of massive amount of data from cities and citizens. Emergence of new digital technologies inspires not only city governments but also city residents, researchers, companies and other stakeholders in discovering and creating new innovative solutions to solve urban challenges and improve peoples' everyday life. Developing novel Internet of Things (IoT) solutions for cities and citizens requires facilities where IoT applications and services can be tested and experimented. The challenge for many smart-city test and experimentation platforms (TEPs), like living labs, has been the lack of sustainable value creation model. This has caused many experimentation platforms to perish after the ending of external funding. A vision about how to construct a robust and continuous IoT test and experimentation platform, as well as instruments for it, is required. The IoT service experimentation platform (IoT SEP) framework presented in this paper provides guidelines for this effort. IoT SEP consist of ten dimensions relevant for establishing a sustainable IoT SEP in smart cities.


I. INTRODUCTION
The notion of smart-city has achieved considerable popularity.Cities around the world experience rapid growth and various technologies are used to solve urban challenges.The contribution of the digital technologies is thought to form the foundation for so-called "smart" cities, but one should not forget the human capital, skilled and educated citizens, sustainable environmental solutions and sustainable economic growth as important attributes for smart urban development [1] [2].Technology like IoT is a viable option to form high quality testing and experimenting platforms in smart cities. Living labs and testbeds are mentioned within the context of various technology experiments in smart cities [5].IoT service experimentation platforms (IoT SEPs) in urban contexts attract numerous stakeholders, including city, research institutions, technology providers and technology users.The reasons for using IoT SEPs vary by stakeholder.For an experimenting company, the ability to evaluate new IoT service concepts with users in real-life environment may yield more accurate test results than alternative settings.The costs of an evaluation cycle will remain lower in experimental platforms.For other stakeholders like cities and research institutions the IoT service experimentation platform provides an opportunity to collect data from real-life settings.That data can later be utilized for improving existing services or developing new services for citizens and other stakeholders in cities. [5] However, technology experimentation platforms in city environments have faced some challenges.Many of them like living labs, appear to last only the time the funding is granted for.One of the several reasons for this is that smart-city experimentation projects with a high integration of community and users result in more sustainable outcomes than technological or infrastructure-driven projects [6].On the other hand, there are examples of successful technology-oriented testbeds, which, having the collaboration of research organizations, government and private sector, appeared attractive to private stakeholders so that they wanted to invest on them, too [7].
For digital platforms, it is crucial to reach the critical mass and open platform interfaces to third parties for the development of a sustainable and expanding platform.A digital platform without open interfaces does not enable the creation of a scalable business platform.[8] By achieving critical mass and opening platform interfaces to third parties, the IoT service experimentation platform may improve value creation and lead to more sustainable performance of IoT SEPs in city context.However, if specialized IoT devices, infrastructure or customer interaction is needed, the scale of IoT SEP may be limited.Thus it is more appropriate to talk about economy of scope instead of economy of scale.
In business networks firms sharing common interests have motivation to develop sustainable relationships that provide mutual benefits [9] [10].These mutual benefits created by the network are also referred to as value networks.A value network might include suppliers, customers and strategic partners providing value to each other.By belonging to value networks, a firm will benefit from the value networks' resources, receiving direct or indirect value [11].Direct value may include goods, services and revenue obtained through material exchange or indirect intangible benefits like knowledge or improved market and networking potential [12] [11].
Establishing a robust IoT service experiment facility and engaging relevant stakeholders in city context is not a trivial task [5] [13].The core value that IoT provides is the information created through connected devices.IoT improves our understanding of the surrounding world, and, at best cases, IoT SEP provides for its stakeholders learning experiences that would otherwise be impossible to have.The prerequisite for testing and evaluating relevant IoT scenarios with real-life (end-) users in real-world environment is mutual trust and commitment between stakeholders.[6] In order to fulfill the expectations and needs of all stakeholders of the IoT SEP, a model for sustainable value creation is required.The purpose of this study is to examine existing testing and experimentation platforms for technology and constitute a framework for evaluating the IoT SEP in smart-city environments.This paper is organized as follows: after the introduction in Section I, Sections II and III review previous literature on TEPs and on value networks.Section IV proposes a framework containing ten dimensions for evaluating IoT SEPs in smart cities.In Section V, the framework is applied in two empirical cases, in the Smart Santander and Smart Kalasatama projects.Section VI discusses the results, and Section VII summarizes the work and draws conclusions.

II. TEST AND EXPERIMENTATION PLATFORMS
A platform is defined "as a reuse of sharing of common elements across complex products or systems of production" [14].Complementing it is a collection of common assets like human and social capital, processes, components, technologies and infrastructures [15] [5].A platform system has three elements; a core, components and the interfaces between them.Platform systems and its components can evolve, but the interfaces remain stable [14].
Four different types of company platforms are identified; transaction, innovation, integrated and investment platforms.A transaction platform is based on a technology, product or service and it acts as an intermediary to facilitate transactions between users, buyers, or suppliers.An innovation platform is based on a technology, product or service providing a ground for other companies to develop complementary technologies, products and services.An integrated platform, combining a transaction and an innovation platform, is not only for exchanging technologies, products or services but also for third-party developers to innovate new technologies, products or services.Investment platform acts as a holding company and/or an active platform for investors.It is established by companies with a platform strategy.[16] ICT has proved to be a significant driving force not only for the emergence of industrial platforms but also for the appearance of test and experimentation platforms in cities [7][5] [17].Industrial ICT platforms like Google, Apple and Microsoft are platforms for products and services, and they provide also a place for external stakeholders to design new complementary technologies, products and services (integrated platform).Reusable common components and technologies form a basis for an industry platform, which is characterized by openness to external parties.The degree of openness related to for example information access, costs of common assets and platform access varies.[17] Network effect is a prominent feature of digital platforms: the more users are engaged with the platform the more attractive that platform becomes for other potential users, and the more users the platform manages to attract the more value the platform generates for its stakeholders.Network effects can be divided into direct and indirect network effects.In direct network effect, users attract and generate new users; in indirect network effect, users attract other platform users or stakeholders, including product or service developers, to join the platform.[16] ICT solutions are often tested on various types of technology-test and -experimentation platforms before final release.These platforms aim to accelerate technologies and innovations and to improve understanding of the socioeconomic changes related to digital technologies and ICT developments [7][5] [6].As complex systems, cities provide multidimensional environments for testing modern ICT technologies.Previous research [7] has paid attention to European broadband open-innovation platforms and distinguished six different test and experiment platform types (Fig. 1.).
Technologies, products or services in the early development phase are tested in prototyping platforms, which are characterized as closed in-house design and development facilities.However, prototyping platform is also utilized as a platform where nearly market ready concepts and new business models are developed and tested in collaboration with other participants.Trust between stakeholders carries significant weight in prototype platforms.Testbed provides a standardized laboratory environment and is used for testing yet immature new technologies, products, services and sometimes even marketing concepts.Risks of the test hazards are minimized in testbeds.Depending on a testbed´s openness with regard to stakeholders, the testbed may induce the creation of new innovative technologies.[7] Field trials are regarded as agile platforms for several stakeholders and even final users.They are used for specific small-scale tests, testing technical features of new technology, product or service in a limited real-life environment.[7] Fig. 1.Test and Experiment platforms [7].
Living labs provide an environment for technology experimentation in real-life context, and their users are integrated to technology innovation process already in the early phase of product development lifecycle.A living lab platform with user involvement provides more contextspecific insights on technology development and acceptance processes, improving mutual stakeholder interaction.In addition, user involvement enables value co-creation and technology co-production with other living lab stakeholders.Experiments run in living labs improve overall understanding of technology impacts and integration to society.Characteristic to living labs is they are run in large-scale contexts such as cities and that their life cycle is rather long.[7][5][6] [18] Market pilots are utilized when a product or a service is close to maturity and ready for commercialization.A market pilot platform is not open, and a product or a service is released for a limited number of end users who will receive market data before the final launch.Finally, the purpose of societal pilots is to introduce a rather mature new product or services in a real-life context.Public involvement in societal pilots is high, as the aim is to produce new societal innovations.[7] As previously mentioned, cities provide a complex realworld environment and thus numerous advantages for various information technology experiments.Many of the test platforms presented by [7] are valid in smart-city context.However, certain attributes like scalability and real-world experiment environment are significant when establishing IoT TEP in smart-city context.City-level experiments enable ICT developers and other stakeholders like research institutions to iteratively asses and validate immature or nearly market ready IoT services and applications.By engaging real users to early IoT services or application development processes the developers also receive valuable hints of usefulness and acceptance of the technology solutions during the iterations.[3][5] Other advantages the cities provide for ICT experimenters are heterogeneous IoT applications and devices in diverse experiment domains.[4] Collecting and monitoring data from traffic, water and energy consumption, air pollution, public buildings and lighting are examples of the diverse city domains where various IoT devices and applications have been implemented and experimented with [18] [19].
The ICT and smart-city IoT TEPs are characterized by openness, user and public involvement, real-world test environment, scalability, IoT heterogeneity and architecture design.Vertical scope, duration and commercial maturity are also mentioned as being TEP characteristics.(Table 1.) Openness here describes the way TEP results are available and open for all stakeholders.Public and user involvement explains how actively policy makers and citizens are involved in TEP activities.Real-world environment implies the way the tests and experiments are implemented in natural and realistic settings and scalability how TEP scales from small to large.IoT heterogeneity describes the diversity of IoT devices and applications.Architecture design illustrates the interoperability of heterogeneous IoT devices and applications and how data is collected and processed.Vertical scope measures the degree to which stakeholders are involved in TEP from different levels in value chain.Duration is related to the degree of TEP stability, and commercial maturity illustrates how close the tested technologies, products and services are to market introduction.

III. VALUE NETWORKS
Firms sharing common interests in business networks have motivation to develop and maintain relationships that provide them mutual benefits [9] [10].The networks are dynamic and may change when a firm deepens its existing relationships, establishes new ones or ends the problematic ones [23] [24].The firm is dependent on resources controlled by other firms.It can compensate for its limited resources, either by developing its position in an existing network, or by establishing new networks [9].The firm's value network may include suppliers, customers and strategic partners who provide value to each other.The value provided can be divided into three categories: 1) goods, services and revenue, as part of material exchange, 2) knowledge regarding the services and technologies and 3) intangible benefits that go beyond actual services and are not accounted in the financial sense [12].In case of IoT SEPs the main stakeholders include the parties listed above, the city, technology providers, companies experimenting with their services and users of those services, among others.The main value received by the experimenting firm is the ability to evaluate new IoT service concepts with users in their every-day environment better than with alternative settings.The relative advantage of IoT SEPs may be due to a faster cycle from product concept to evaluation, lower cost of an evaluation cycle or set-up, better fit, higher number of experimental users available, better research support for experimentation or some other benefits provided by the experimentation platform or the other stakeholders in the IoT SEP value network.The value provided by the experimenting firm may be material or intangible benefits related to exchanged value, such as the interest of users to experiment with new products, the interest of a partner to gain knowledge of a new technology or the interest of a public party orchestrating a network to activate new business development in an IoT SEP.
Many experiment platforms, especially living labs, have suffered from the lack of sustainable value creation model after project funding has been used up to set up the platform.Following the logic of platform economics [17], a platform enterprise orchestrating a value network has to use resources to establish and maintain the network or platform.If the value network orchestrator is a public organization in a PPP setting, the motivation to use public funding beyond the project is limited.A sustainable value network requires sufficient volume of value creation throughout the network to maintain a healthy network.This is also referred to as ecosystem health [25] and includes productivity, robustness and niche creation components.

IV. FRAMEWORK FOR IOT SERVICE EXPERIMENT PLATFORM -
DIMENSIONS TO EVALUATE IOT SEP Cities are responsible for their residents´ wellbeing, and therefore cities have many functions and activities that must work together without interruptions.This sets high requirements for city governance, management and ICT architecture.[14][13] ICT plays a significant role in city development: the city´s ability to collect data from various sources and to analyze data to improve diverse city services turns a traditional city into a smart one [13] [5].As a complex system cities provide a multifaceted foundation to test and experiment new digital technologies and ICT solutions in reallife user context [7].Numerous smart-city IoT test and experiment initiatives demonstrate good results, but the challenge is the sustainability of TEPs.It seems the integration of the community and users to technology experiments improves the duration of technology TEPs [6].On the other hand, the lack of sustainable value creation and lack of business model may influence negatively the technology TEPs` duration in smart cities [20].Regarding the TEP continuity and importance of sustainable value creation model, city authorities need instruments for creating and developing such IoT SEPs that are attractive enough for external stakeholders to participate in IoT SEP activities.Fig. 2 presents a framework with ten dimensions, which are the elements significant for establishing a sustainable IoT SEP ecosystem.The dimensions extended from the TEP characteristics presented in Table 1 are scaled from 0 to 3. Value 0 represents no activities, value 1 moderate, value 2 good and value 3 excellent performances.Openness (Fig. 2) is related to the degree the stakeholders have access to certain assets of IoT SEP.These assets may include facilities, information, infrastructure, technologies or user communities.Openness also describes the level an IoT SEP is integrated to other test and experiment facilities.[7][5] Real-world environment indicates the level at which the experiments are executed in natural and realistic environment [3][4][5] [18] [19] .Public and user involvement, which describes the level to which city authorities and citizens are involved in IoT SEP, is vital as cities are commonly regarded as the primary initiators and enablers of smart-city experiments.Therefore, the interest of city administrators and their involvement in IoT SEP is significant.Recently, the users' and especially the end-users' role in smart-city IoT experimentation have been highly emphasized.By integrating users in the early phase of product development and testing, the IoT developers and researches can receive instant feedback of their product or service.This information is valuable as it provides data for further development of IoT services and improves understanding of technology acceptance and social impact of IoT solutions.[5][6] [7] Vertical scope describes the degree to which stakeholders from different stages of value chain are involved in TEP, from technology providers to end users.Vertical and horizontal scope also describes how IoT SEP is integrated to other test and experiment platforms.[7][5] Many smart-city initiatives have been created around a vertical industry or an industry emphasizing the goals of a single vertical theme, such as energy efficiency, traffic or health care.The vertical approach will also influence the choice of employed technologies and standards that will best support the needs and requirements of a chosen industry.Horizontal scope describes TEPs´ wider set of applications and services combining data from multiple industries to service developers.That approach challenges and associates costs on finding a balanced set of technologies while matching the requirements for all the verticals targeted.However, potentially it can reach a wider user base for services based on new services.
In order to receive extensive and deep data in city context, IoT experiments need to be executed in large-enough scale with real users in real environment.The scope may vary from a laboratory environment to a living lab providing scalability to test and experiment with IoT services.Platform scalability creates more value and attracts more users.In digital platforms, scalability is regarded as an outcome of initial success and together with network effect acts as a foundation for further growth.IoT SEP can act as a catalyst for this through implementing an efficient and effective experimentation process that provides significant added value for service and product development firms.[7][16] The challenge of TEPs in city context has been the lack of sustainable value creation and continuity once the funding has dried out.The more an IoT SEP is able to attract users the more value is generated through direct or indirect network effects.This will provide more opportunities for co-creation and evaluation of new ideas and scenarios in collaboration with users: citizens, research institutions, companies, city administrators and other interest groups.This also will enhance the sustainability and continuity of IoT SEPs.[5][6][18] [16] IoT heterogeneity and architecture design illustrates the diversity of IoT devices and applications and interoperability between them.The level of architecture design also indicates a city´s ability to design and manage complex data sources and streams from various domains.Kalasatama provides a ground for external companies to pilot new digital services [27].Both Smart Santander and Kalasatama provide an experiment platform for researchers, companies, application and service providers, user communities and city administrations.Smart Santander and Kalasatama are monitored and the data is collected through websites, project open calls and academic papers.[26][27] Here we evaluate them through ten dimensions of IoT SEP presented earlier in this paper in (Fig. 2).The grading of evaluation runs from 0 to 3 as explained in Section IV.
The Smart Santander project is divided into four subsystems: AAA (authentication, authorization and accounting), testbed management, experimental support and application support.Smart Santander is open for domestic participants, but the project has been deployed also in Belgrade/Pancevo in Serbia, Guildford in the UK and Lübeck in Germany.Two open calls for European project partners were issued during 2012 -2013.The purpose of the open calls was to attract exciting experiments, experimentally-driven scientific research with high impact and evaluations that utilize features provided by the Smart Santander facility.Both open calls were targeted to public and private R&D organizations with expertise in smart cities and IoT.The first project period was for one year in 2012 and the second for six months in 2013.The first call covered 3-5 experiments with 1-2 partners for each experiment.As a result, nearly 50 proposals, of which half came from industry partners, were received.After the first call, 4 new partners were selected.Second call included 4-6 experiments for 1-2 partners per experiment.Smart Kalasatama is a district of Helsinki in Finland.It is estimated that by 2035 around 21,000 inhabitants will live in the area, which provides job opportunities for 8,000 people.Sustainability and business-friendly environment are the driving forces of the area.Smart Kalasatama is a partner for the bIoTope project in aiming to build an IoT OPen innovation Ecosystem for connected smart objects.BIoTope is implemented in Lyon, Helsinki and Brussels regions.The first pilots executed in Smart Kalasatama included smart-mobility services, effective waste management, food waste reduction and co-creation of local services.The next call for new innovative pilots is targeted for health and wellbeing services.Kalasatama pays 1,000-1,800 euros for each pilot carried out there.The purpose of the pilots is to provide service experience for citizens.

A. Openness
The aim of the 20,000 sensors in four cities (Santander, Belgrade/Pancevo, Guildford and Lübeck) is to provide realworld data for developing novel applications and services for city and citizens.Smart Santander is open for research institutions and companies providing additional benefits for other stakeholders in Smart Santander project (Fig. 3.).

Kalasatama belongs to the 6aika project, which aims to provide a platform for open innovations, open data and interfaces, as well as open participation and customership.
There are six Finnish cities participating in the project.Their partnership with cities of Lyon and Brussels through the IoT OPen innovation Ecosystem (bIoTope) project extends the interfaces for data and IoT solutions (Fig. 4

B. User/Public involvement
City administration and citizens are significant actors in Smart Santander and Kalasatama.The primary task of the applications and services is to create value for city authorities and citizens in order to improve their lives.Public governments are enablers and citizens active participants both in Santander and Kalasatama.

C. Vertical and Horizontal scope
The current Smart Santander infrastructure targets transportation, energy and environment.The goal is to extend the infrastructure to cover new application domains.One aim would be to create a mechanism to evaluate end-user feedback or quality of experience automatically or semi-automatically during experimentation.Kalasatama provides a living lab facility for agile pilots.Its pilots have included smart-mobility services, effective waste management, health and wellbeing services and co-creation of local services.

D. Scalability
Scalability in the Smart Santander project varies from indoor to citywide experimentations.Smart Santander is implemented in four European cities.With 12,000 sensors, Santander forms the widest real-life IoT experimentation platform in the Smart Santander project.Guildford functions as an indoor test environment with 250 sensor nodes.The Lübeck facility consists of 300 stationary sensors and provides information for commercial use, for example bus timetables and locations.The EkoBus system is deployed in Belgrade and Pancevo.The system utilizes public transportation vehicles to monitor environmental parameters such as temperature, humidity and CO2 information for city authorities and information such as arrival times and location of the buses for end users.The system utilizes 60 devices, but there is no direct access to any IoT nodes.Smart Kalasatama pilots are implemented in the Kalasatama district, which is connected through partnership projects to other smart-city platforms in Finland, Belgium and France.

E. Sustainable value creation
The objective of both Smart Santander and Smart Kalasatama is to produce innovative IoT applications and services which clearly demonstrate benefits and value for the city, its residents and stakeholders.As an example, SmartSantanderRA application is being developed to provide information on nearly 3,000 places in Santander.The application provides real-time data on traffic, weather and bike rental services, for example.For the European researcher community, the Smart Santander infrastructure provides a platform to carry out scalable IoT experiments in a real-life environment.Agile pilots in Smart Kalasatama have produced applications and digital services which improve food and waste management, and neighborhood aid services as wella neighborhood aid platform, NappiNaapuri, connects Kalasatama residents who need aid in everyday situations.

F. Continuity
The Smart Santander project received funding of around 10 million euros from EU to carry out large-scale technology testing in Santander city.The funding was granted for four years, starting from 2010.As the sensors are permanently placed around the city, data is still being collected even though the funding period is over.The data is used globally for research purposes and locally to develop services and applications.The Smart Kalasatama infrastructure built for agile pilots provides an environment for continuous piloting of products and services.

G. IoT heterogeneity
Smart Santander offers a diverse set of IoT devices and systems.The IoT nodes consist of actuators, sensors, QR and NFC tags and mobile-phone-based sensing platforms which are connected via various network technologies.The aim is to deploy around 20,000 actuators, tags and sensors around the partner cities in Europe.[4] Smart Kalasatama is a new district being developed in Helsinki.Residential buildings have their own data centers, where for example energy consumption data is collected.Moreover, a broader, heterogeneous IoT network is evolving, and it is expected that diverse IoT systems will emerge.

H. Architecture design
The architecture design in the Smart Santander platform is based on three-layered architecture.The layers consist of IoT device, IoT gateway and server levels.Most of the experimental IoT devices are at the IoT device level.The IoT gateway node level links the IoT devices at the edges of the network to the core network infrastructure.The server level consists of server devices which are connected to the core network infrastructure.The architecture design is well-planned and implemented.[4] Information of the architecture design from Smart Kalasatama was not available in public sources, and thus could not be evaluated.However, the absence of public data implies that no open architecture design has been deployed so far.

VI. DISCUSSION
Technology test and experimentation projects in smart-city environments tend to decline once project funding has finished.Some studies show that integrating communities and users to ICT experimentation platform activities results in more sustainable outcomes.Another factor effecting platform sustainability is the critical mass.Achieving a critical mass requires large-enough user base to make the platform attractive for other users and external service developers and providers.Wide horizontal and vertical scope, i.e. the ability to have stakeholders from multiple industries and from technology providers to end users will increase the number of participants in the IoT SEP.The smart-city experimenting platform should therefore provide open interfaces, share assets, create efficient and effective experimentation processes and be compatible with other city platforms.
Reaching the critical mass results in network effect, which is one of the success indicators of sustainable digital platforms and a turning point for a platform´s continuity and growth towards a self-sustaining platform.In the case of Smart Santander and Kalasatama, the technology experimentation platform is extended from the original region to other European cities.As the end-users of the Smart Santander and Smart Kalasatama platforms are the citizens, they enable experiments in real-life environment with real users to be carried out.Smart Santander makes possible both small and large-scale experiments with its partners.Smart Santander collaborates widely not only with research institutions in Europe but also with research institutions outside Europe.Smart Kalasatama has provided a platform for several agile pilots producing services valuable for citizens.Later on, it will benefit from collaboration with other cities in Finland and Europe.Both Smart Santander and Smart Kalasatama have attracted various application and service developers.However, it remains unclear how many service providers and companies have been involved in Smart Santander and how many new innovations Smart Santander has generated.Smart Kalasatama attracted 52 offers after the first call for agile pilots.Four pilots were selected to carry out the prototype tests in Kalasatama.Applications for mobility services and social web services were developed to improve everyday lives It appears that Smart Santander has managed to create a large-enough user community, open interfaces and effective and efficient experimentation processes, so that the stakeholders such as city administration, global scientific community and private sector are showing great interest in maintaining the Smart Santander experiment platform.This supports the argument about the importance of critical mass in the creation of compatible and open platforms with other platforms in city context.However, the network effect and revenue model for the Smart Santander innovation platform remains unclear.Smart Kalasatama is taking just the first steps with technology experimentation platform activities; nevertheless, the results of the agile pilots have been valuable and encouraging.It is expected that Smart Kalasatama will produce innovative and beneficial applications and services in collaboration with the partner cities in Finland and Europe.

VII. SUMMARY AND CONCLUSION
One of the main challenges of test and experimentation platforms in city context has been the lack of sustainable value creation models.This has resulted in the discontinuity of experimentation platforms after the external funding has ended.It is argued that obtaining a critical mass with real-life users and providing open and compatible platforms in city context improve sustainability and value creation of IoT SEPs in smart cities.This critical mass can be supported best if the platform scope covers multiple industries and stakeholders from technology providers to end users, i.e. a wide horizontal and vertical scope.Altogether, we were able to identify eight relevant factors from the previous literature and, in this paper, we extend the list with sustainable value creation and horizontal scope.Out of these, we constructed and presented a framework consisting of ten dimensions essential for sustainability of service experimentation platforms in smartcity environments.
The dimensions are openness, real-world experiments, user/public involvement, vertical and horizontal scope, scalability, sustainable value creation, continuity, IoT heterogeneity and architecture design.These dimensions represent the significant elements for establishing a sustainable IoT SEP and a robust ecosystem around it.
For practitioners, this framework provides a means to evaluate and guide the development of IoT SEPs.For researchers, the framework provides insights for future research on IoT SEP ecosystems and value formation in city context.In addition to these, we call for further research on relevant business models for IoT SEPs in smart-city context.

Fig. 2 .
Fig. 2. A framework to evaluate IoT service experiment platform in smart-city.
V. SMART SANTANDER AND SMART KALASATAMAEXPERIMENT  PLATFORMSIn this section two smart-city projects, Smart Santander in Spain and Smart Kalasatama in Finland, are evaluated through the IoT SEP framework presented in Fig 2.These projects were selected because they provide large-scale real-world experiment environments, aiming to deploy IoT infrastructure around Santander, Kalasatama and partner cities.The citizens and visitors of Santander and Kalasatama are integrated as active users in IoT services.SmartSantander´s target is to install 12,000 sensors, actuators and tags around Santander and 8,000 sensors to other European partner cities in order to collect environmental and traffic data, among other data.[26]