A Novel Spatio-Temporal Data Storage and Index Method for ARM-Based Hadoop Server

Laipeng Han, Lan Huang, Xueyi Yang, Wei Pang, Kangping Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)
5 Downloads (Pure)

Abstract

During the past decade, a vast number of GPS devices have produced massive amounts of data containing both time and spatial information. This poses a great challenge for traditional spatial databases. With the development of distributed cloud computing, many highperformance cloud platforms have been built, which can be used to process such spatio-temporal data. In this research, to store and process data in an effective and green way, we propose the following solutions: firstly, we build a Hadoop cloud computing platform using Cubieboards2, an ARM development board with A20 processors; secondly, we design two types of indexes for different types of spatio-temporal data at the HDFS level. We use a specific partitioning strategy to divide data in order to ensure load balancing and efficient range query. To improve the efficiency
of disk utilisation and network transmission, we also optimise the storage structure. The experimental results show that our cloud platform is highly scalable, and the two types of indexes are effective for spatio-temporal data storage optimisation and they can help achieve high retrieval efficiency.
Original languageEnglish
Title of host publicationCloud Computing and Security
Subtitle of host publicationSecond International Conference, ICCCS 2016. Lecture Notes in Computer Science
EditorsX. Sun, A. Liu, H.-C. Chao, E. Bertino
PublisherSpringer
Pages206-216
Number of pages11
ISBN (Electronic)978-3-319-48671-0
ISBN (Print)978-3-319-48670-3
DOIs
Publication statusPublished - 2016
EventSecond International Conference, ICCCS 2016 - Nanjing, China
Duration: 29 Jul 201631 Jul 2016

Publication series

NameInformation Systems and Applications, incl. Internet/Web, and HCI (Lecture Notes in Computer Science)
PublisherSpringer
Volume10039

Conference

ConferenceSecond International Conference, ICCCS 2016
CountryChina
CityNanjing
Period29/07/1631/07/16

Fingerprint

Cloud computing
Servers
Data storage equipment
Electric power transmission networks
Resource allocation
Global positioning system

Cite this

Han, L., Huang, L., Yang, X., Pang, W., & Wang, K. (2016). A Novel Spatio-Temporal Data Storage and Index Method for ARM-Based Hadoop Server. In X. Sun, A. Liu, H-C. Chao, & E. Bertino (Eds.), Cloud Computing and Security: Second International Conference, ICCCS 2016. Lecture Notes in Computer Science (pp. 206-216). (Information Systems and Applications, incl. Internet/Web, and HCI (Lecture Notes in Computer Science); Vol. 10039). Springer . https://doi.org/10.1007/978-3-319-48671-0_19

A Novel Spatio-Temporal Data Storage and Index Method for ARM-Based Hadoop Server. / Han, Laipeng; Huang, Lan; Yang, Xueyi; Pang, Wei; Wang, Kangping.

Cloud Computing and Security: Second International Conference, ICCCS 2016. Lecture Notes in Computer Science. ed. / X. Sun; A. Liu; H.-C. Chao; E. Bertino. Springer , 2016. p. 206-216 (Information Systems and Applications, incl. Internet/Web, and HCI (Lecture Notes in Computer Science); Vol. 10039).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Han, L, Huang, L, Yang, X, Pang, W & Wang, K 2016, A Novel Spatio-Temporal Data Storage and Index Method for ARM-Based Hadoop Server. in X Sun, A Liu, H-C Chao & E Bertino (eds), Cloud Computing and Security: Second International Conference, ICCCS 2016. Lecture Notes in Computer Science. Information Systems and Applications, incl. Internet/Web, and HCI (Lecture Notes in Computer Science), vol. 10039, Springer , pp. 206-216, Second International Conference, ICCCS 2016, Nanjing, China, 29/07/16. https://doi.org/10.1007/978-3-319-48671-0_19
Han L, Huang L, Yang X, Pang W, Wang K. A Novel Spatio-Temporal Data Storage and Index Method for ARM-Based Hadoop Server. In Sun X, Liu A, Chao H-C, Bertino E, editors, Cloud Computing and Security: Second International Conference, ICCCS 2016. Lecture Notes in Computer Science. Springer . 2016. p. 206-216. (Information Systems and Applications, incl. Internet/Web, and HCI (Lecture Notes in Computer Science)). https://doi.org/10.1007/978-3-319-48671-0_19
Han, Laipeng ; Huang, Lan ; Yang, Xueyi ; Pang, Wei ; Wang, Kangping. / A Novel Spatio-Temporal Data Storage and Index Method for ARM-Based Hadoop Server. Cloud Computing and Security: Second International Conference, ICCCS 2016. Lecture Notes in Computer Science. editor / X. Sun ; A. Liu ; H.-C. Chao ; E. Bertino. Springer , 2016. pp. 206-216 (Information Systems and Applications, incl. Internet/Web, and HCI (Lecture Notes in Computer Science)).
@inproceedings{1bc9f85b2e474d2f9dc5d302e350755c,
title = "A Novel Spatio-Temporal Data Storage and Index Method for ARM-Based Hadoop Server",
abstract = "During the past decade, a vast number of GPS devices have produced massive amounts of data containing both time and spatial information. This poses a great challenge for traditional spatial databases. With the development of distributed cloud computing, many highperformance cloud platforms have been built, which can be used to process such spatio-temporal data. In this research, to store and process data in an effective and green way, we propose the following solutions: firstly, we build a Hadoop cloud computing platform using Cubieboards2, an ARM development board with A20 processors; secondly, we design two types of indexes for different types of spatio-temporal data at the HDFS level. We use a specific partitioning strategy to divide data in order to ensure load balancing and efficient range query. To improve the efficiencyof disk utilisation and network transmission, we also optimise the storage structure. The experimental results show that our cloud platform is highly scalable, and the two types of indexes are effective for spatio-temporal data storage optimisation and they can help achieve high retrieval efficiency.",
author = "Laipeng Han and Lan Huang and Xueyi Yang and Wei Pang and Kangping Wang",
note = "This project is supported by Science and Technology Development Plan of Jilin Province (20140204010SF) and Chinese National Natural Science Foundation (61472159). WP is supported by the PECE bursary from The Scottish Informatics and Computer Science Alliance (SICSA).",
year = "2016",
doi = "10.1007/978-3-319-48671-0_19",
language = "English",
isbn = "978-3-319-48670-3",
series = "Information Systems and Applications, incl. Internet/Web, and HCI (Lecture Notes in Computer Science)",
publisher = "Springer",
pages = "206--216",
editor = "X. Sun and A. Liu and H.-C. Chao and E. Bertino",
booktitle = "Cloud Computing and Security",

}

TY - GEN

T1 - A Novel Spatio-Temporal Data Storage and Index Method for ARM-Based Hadoop Server

AU - Han, Laipeng

AU - Huang, Lan

AU - Yang, Xueyi

AU - Pang, Wei

AU - Wang, Kangping

N1 - This project is supported by Science and Technology Development Plan of Jilin Province (20140204010SF) and Chinese National Natural Science Foundation (61472159). WP is supported by the PECE bursary from The Scottish Informatics and Computer Science Alliance (SICSA).

PY - 2016

Y1 - 2016

N2 - During the past decade, a vast number of GPS devices have produced massive amounts of data containing both time and spatial information. This poses a great challenge for traditional spatial databases. With the development of distributed cloud computing, many highperformance cloud platforms have been built, which can be used to process such spatio-temporal data. In this research, to store and process data in an effective and green way, we propose the following solutions: firstly, we build a Hadoop cloud computing platform using Cubieboards2, an ARM development board with A20 processors; secondly, we design two types of indexes for different types of spatio-temporal data at the HDFS level. We use a specific partitioning strategy to divide data in order to ensure load balancing and efficient range query. To improve the efficiencyof disk utilisation and network transmission, we also optimise the storage structure. The experimental results show that our cloud platform is highly scalable, and the two types of indexes are effective for spatio-temporal data storage optimisation and they can help achieve high retrieval efficiency.

AB - During the past decade, a vast number of GPS devices have produced massive amounts of data containing both time and spatial information. This poses a great challenge for traditional spatial databases. With the development of distributed cloud computing, many highperformance cloud platforms have been built, which can be used to process such spatio-temporal data. In this research, to store and process data in an effective and green way, we propose the following solutions: firstly, we build a Hadoop cloud computing platform using Cubieboards2, an ARM development board with A20 processors; secondly, we design two types of indexes for different types of spatio-temporal data at the HDFS level. We use a specific partitioning strategy to divide data in order to ensure load balancing and efficient range query. To improve the efficiencyof disk utilisation and network transmission, we also optimise the storage structure. The experimental results show that our cloud platform is highly scalable, and the two types of indexes are effective for spatio-temporal data storage optimisation and they can help achieve high retrieval efficiency.

U2 - 10.1007/978-3-319-48671-0_19

DO - 10.1007/978-3-319-48671-0_19

M3 - Conference contribution

SN - 978-3-319-48670-3

T3 - Information Systems and Applications, incl. Internet/Web, and HCI (Lecture Notes in Computer Science)

SP - 206

EP - 216

BT - Cloud Computing and Security

A2 - Sun, X.

A2 - Liu, A.

A2 - Chao, H.-C.

A2 - Bertino, E.

PB - Springer

ER -